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These three documents (final performance report, 
evaluation report, methodology guide) rre products of a project 
through which the Career Advancement Center provided workplace 
literacy services to two industries in Greenville, South Carolina. 
The final performance report discusses achievement of objectives, 
specifically employee assessment, training program development, 
literacy training, offering of support services, establishment of 
evaluation systems, evaluation of training impact, and evaluation of 
project methodology. Other contents include a copy of a Skill Rating 
packet, a sample showing the compiled data at one classroom site, 
project schedule, and tables showing participant numbers and 
characterir>tics . The final evaluation report describes the purpose of 
the evaluation, the project, and the evaluation method, a modified 
version of the Context-Input-Process-Product model. These findings 
are reported: strong evidence that programs implemented and delivered 
services to a minimum of 150 workers at each industry site; only a 
moderate amount of evidence of successful program implementation and 
offering of additional services; and no evidence of development of a 
training methodology meeting specific workplace needs • The 
methodology guide contains a list of project goals and activity 
flowchart. These steps in program implementation are explained: needs 
analysis or job task analysis, curriculum development, employee 
orientation, assessment, training, support services, and training 
completion. (YLB) 
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T.O.P. 

(TRAININS OPPOTTUNITIES PARTNERSHIP) 
GRANT OBJECTIVES 



OBJECTIVE Is To reduce the incidence of ^rkplace illiterar/ in Greenville 
County by establishing a partnership with three area coo|)anies that will provide 
for the assessoaent of at least 200 emploj^s of each company during the 
eighteenHDBonth project period. 

During the grant proposal phase, a partnership was established with three 
companies agreeing to participate in the project: JPS Monaghan Plant, Homelite, 
and Cryovac. prior to grant apporval, Cryovac chose an alternative plan of 
basic skills education for its eirployees and withdrew from the cooperative 
venture with Greenville Technical College. This sequence of events allowed the 
project to expand to additional manufacturing sites, resulting in a partnership 
that involved five industries: 



JPS Monaghan Plant 
JPS Parker Plant 
JPS Slater Plant 
JPS Dunean Plant 
Homelite Corporation 

An orientation program was provided at each industry to acquaint ertployees with 
the project and with the educational opportunity being offered. From this 
orientation, employees would decide of their interest in participating. Some 
companies required assessment of all eirployees, regardless of their interest in 
participating in the educational program; others provided the assessment option 
for only those selecting to enter the educational program. A total of 2257 
eitployees attended the program orientation; 1134 individuals coitpleted tEe" 
assessment phase of the project. A breakdovmTol participation by site follows: 

Orientation 
Homelite 342 
JPS Dunean 239 
JPS Monaghan 197 
JPS Slater 436 
JPS Parker 1043 



Assessment 
409 
239 
130 
119 
237 



OBJECTIVE 2: To develop a training methodology for adult learners to meet 
specific workplace needs. Training materials will be individualized for each 
pamership industry and will utilize that industy's available job^related 
materials. 



A training program was developed and inplemented utilizing the followinq three 
components: 

(1) Literacy tutoring: a one-on-one individualized program provided through the 
Greenville Literacy Association Individuals who were assessed as having 
reading skills below the fifth grade level were given the educational option of 
individual tutoring, with this program, a tutor is matched to the student and 
his specific needs. The tutoring schedule is worked out between the tutor and 
student and instruction takes place at a location agreed upon by both 
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(2) GED Preparatory Instr . ..^on: This component of the project was offered 
on-site to those interested in readying themselves for GED testing. Instruction 
was provided through the Adult Education Division of the School District of 
Greenville County. 

(3) Basic Skills: A job-specific curriculum was developed for each industry that 
would provide up-grading in the math and reading skills utilized in the 
workplace. Involvement of employees in the curriculum development was achieved 
by the use of DACUM panels that were coirprised of both managerial and hourly 
personnel. Their work resulted in the creation of a training needs matrix that 
provided curriculxmi developers with a basis for program content. Where 
necessary, on-site interviews and observation were added to insure that 
job-related materials and procedures wore a fundamental part of the program. 



OBJECTIVE 3: To offer the developed literacy training to at least 150 enployees 
of each pamership industry during the project period. Uie specific training 
suggested for each targeted enployee will be based on that employee's initial 
assessoieat scores. 

It can be concluded that all who attended the T.O.P. orientation programs were 
offered the opportunity to participate. Those selecting to take part in the 
assessment phase of the program were given a more individualized invitation. 
In returning assessment results to each eitployee, a project counselor provided 
not only the test scores, but also the program options roost appropriate for that 
individual. This one-on-one encounter allowed the employee a thorough 
tunderstanding of his needs and how the program could assist. Figures showing 
participation in both program orientation and assessment follow: 



Orientation Assessment 
Homelite 342 409 

JPS Dunean 239 239 

JPS Monaghan 197 130 

JPS Slater 436 ^ 119 

JPS Parker 1043 237 



OBJECTIVE 4: Td offer additional services to each project participant throu^ 
the use of an individualized program of literacy instructicxi that is supported 
by personal counseling, individual and/or grou^ tutorial services, child care, 
and/or transportation assistance. Hiese additiCHial support services are 
expected to increase the average course completion rate from 50 percent to at 
least 60 percent or more;. 



The following support services were offered: 

(1) Counseling: The role of the counselor began with the assessment process. 
As assessment results were provided and enrollments accepted^ a counselor was 
assigned to each student. That counselor would assist in establishing 
individual goals for the student at program outset. Interim visits to the 
classroom ensured continual observation of the student and encouragement in 
successful participation. Where absenteeism or other problems arose, plant or 
phone contacts were made and appropriate assistance provided. 
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(2) Childcare: Assistance with the payment of childcare expenses incurred 
during class time was made available to all program participants. By indicating 
their need for this service and providing their counselor with the childcare 
provision information needed, an employee could secure payment for the child 
care provider on a monthly basis. 

irHo^T^^^^^^'''^'* students needing transportation assistance in 

order to attend classes, bus passes were provided on a monthly basis. Access to 
each student "^^ ^^^"^ ^9^^", through the program ■ counselor assigned to 

These support services were offered to ensure that traditional barriers to 
S^^;H,K'h'°" T eliminate those interested in pursuing this educational 

opportunity. It was assumed that the course completion rate would be improved 

T n^* 'Sf °^u?^fno^^'^'^*?- ^ °^ 513 employees enrolled in the 

T.O.P. Of this number 192 were placed in Literacy Tutoring . The remaining 321 
were involved m either GED Preparation or Basic skills , g ertifir.f.; "f^ — 
completion were awarded to ibb ot these students, iESmFig a completion rate of 
4|%. All of these students attended, however, on a voluntary Lsis, with no 
coK?)ensatory or on-shift time provided. 

differpi^h^ro^*''h?^f'°'" ^^^"^ the significant 

S^fn^r f ^ on-shift course time can make in attendance. One project 
partner saw a specific need for Pre-SPC Math . The program materials were 
developed and provided by the project and classes offered. ^ toJal 0^70 
students were served in approximately a 12-week time span, with 9 hours-5f 

Ja^'^f apprS!nltKy-95_?^' « -"^^^^ion 

J^tP^h^ concluded that the support services had less impact on the completion 
lil t"-^ offering compensatory or on-shift time for participation, with 

"^"^^^ ^ "T-O-P- P'^og'^am, child care knd 
^T^^ assistance were not critical needs. Counseling did orovide an 

added layer of support and encouragement, but did not always Lke the difference 
^^tPnd^n^''''?^ c^'°PP^''^-2Vt or staying with the program^ S tS^ co^iSt 
of attending class is significant and allowing the use of on-shift time for 

SfsSpS^r^e^r^KeT' '° ^ ''''''' ^° compleUoVtran'^^re 

raOECTIVE 5: Tto measure the impact of training on project participants throuah 
an evaluative process of tracking skill achievlment, job prSuctSit? and 
personal growth and development. t"^ouuccivii,y, ana 

^mSs!°" '^'^^"^ "^'^ established to measure the impact of training in three 

(1) Skill level achievement; (2) Job productivity; (3) Personal growth and 

SisCof -thf fn??'^^''r °'k^^=^ evaluation Jysiem is contained Se 
analysis of the following two objectives. 
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CBJECnvE 6: To evaluate the impact of project training on the partnership 
industries through an assessment by nanagement ancVor supervisory personnel of 
individual employee productivity. 

In order to secure data that would assess the impact of training on job 
productivity, workshops were held with supervisory personnel and the project's 
external evaluator to identify appropriate evaluation factors. Resulting from 
these workshops was a Job Performance Skill Rating that was completed for each 
student (by his supervisor) both at program entry and exit. A copy of the Skill 
Rating packet and a sample showing the compiled data at one classroom site is 
included. Analysis of the data is found in the external evaluator 's report. 

OBJECTIVE 7s TO evaluate the project methodology by determining original 
literacy levels of project participants through initial assessment scores and 
comparing these scores to post-training test results. 

Data that would measure the impact of training on the literacy levels of project 
participants was secured by utilizing two methods of evaluation 

(1) A Student Record Sheet was maintained for each basic skills 'participant to 
record pre-and post- test scores as well as attendance data. 

(2) A Skill Rating Analysis was completed on each student by the classroom 
instructor. This analysis measured the impact of training on math skills." 
reading and language arts skills and self-esteem. A copy of each of the 
evaluation instruments is included, as is a sample showing the data collected at 
one classroom site. An analysis of the data is found in the external 
evaluator 's report. 



S?'^!^ J^yFl^^^.^ t ^^"^ project report on tlie laethodology and results 

available for dissemination to interlJted gro^ Lj 
instituticMns statewide and nationwide. 

'^l ^rU^i l^'^rl °" program methodology was developed as a separate product of 
S^^?^h?r^' ■ I i^^luded as part of the final report and has beefi made 
available to interested groups and institutions statewide and nationwide 



Job Performance 
Skill Rating 



I. Dependability 



4 2 



5 

Perfect attendance 


3 

On time and ready for 
work 


1 

Constantly late 


Always meets or exceeds 
quaiiiy stanaarus 


Regular attendance 


Frequently absent 


Volunteers for overtime 


Stays on Job 


Never wants to work 
overtime 


Accomplishes more than 
what he says he will 


Will work overtime 
when asked 


Promises but does not 
deliver 




Quality of work within 


Does not meet 
proaucuon/ quality 
standards 




Accomplishes what he 
says he will 






Does what is required 
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Job Performance 
Skill Rating 



II. Teamwork 



4 2 



5 

Promotes team spirit in 
department to achieve 
goals 


3 

Offers to assist others 
when asked 


1 

Refuses to work with 
others in department 


Makes constructive 
suggestions 


Enjoys and gets along 
with others 


Complains constantly 


Accepts challenges 


Demonstrates concern 
for others' feelings 


Does not enjoy work or 
working with others 


Constantly seeking to 
improve processes 


Works independently; 
participat ^s only as 
requested 


Wastes time 


Voluntarily gets involved 
with others' nroblems 


Tries to participate in 

\X\y^aJ. LilXCliLdl CSIIUI Lo 


Blames others for 
pioDiems 


Takes charge as a leader 


Reports things that are 
amiss to others 






Has few comments; just 
does his Job 






Goes along with others* 
suggestions 





Job Performanc 
Skill Rating 



m. SelfXniative 



4 


[ 2 


5 

Does not wait to be told 
what to do 


3 

Good attitude; a steady 
employee 


1 

Usually fails to take 
charge of assigned 
task(s) 


Initiates new ideas or 
ways to improve 
processes 


Self-wStarter unless it is 
a new or different job 
task 


Always waits to be told 
what to do next 


Assists without being 
asked 


Learns all aspects of 
current job duties 


Always exerts minimal 
effort 


Requires little or no 
supervision 


Makes adjustments to 
new ideas and changes 
with minimal 
resistance 


Never tries to advance 


Requests experience 
beyond scope of present 
Job duties 


Content, comfortable 
with current Job duties 


Poor attitude 




Follows instructions 


Shies away from 
opportunities to learn 
new Jobs 






Exhibits or expresses 
some distrust of 
management 



Job Performance 
Skill Rating 



IV. Job Knowledge 



4 2 



5 

Assesses progression of 
work ia regards to 
quality and quantity 
standards 


3 

Relates problems to 
supervisor 


1 

Unable to asses 
progression of work in 
regards to quality and 
quantity standards 


Assists other workers 


Performs routine Job 
tasks well; mals:es 
minimal errors 


Always needing help 


Demonstrates 
exceptional math and 
reading skills on the job 


Demonstrates good 
math £ind reading skills 
on the Job 


Runs defective paits 
knowingly 


Makes suggestions for 
improving processes or 
solving problems 


Expresses or exhibits 
desire to progress to 
new areas ot 
responsibility 


Often reacts instead of 
acting 


Exceeds requirements of 
Job performance 


Is knowledgeable about 
Job 


Makes frequent errors 
while using job reading 
and math skills 


Can explain job duties 
and tasks to others 


Is easily trained to 
perform new tasks 


Fails to upgrade 
technical skills 


Works with supervisor to 
solve problems 


Adapts to changes in 
procedures 


Does not understand 
what to do 




Follows quality 
procedures 


Makes frequent errors 
while performing Job 
tasks 






Requires additional 
training on new or 
changed procedures or 
tasks 
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V. Adaptability 



4 2 



5 

Performs unusual tasks 
willingly 


3 

Cooperates as needed; 
performs unusual or 
new tasks with 
supervision 


1 

Complains when asked 
to do something out of 
the ordinary 


Demonstrates 
acceptance of change 
and works hard to see 
that it succeeds 


Helps others without 
complaining 


Performs unacceptable 
work on any unusual or 
new tasks 


Performs less desirable 
tasks willingly 


Demonstrates 
reluctance, skepticsim 
aoout cnange, snows 
preference to continue 
known tasks 


Exhibits or expresses 
reluctance to take on 
any extra worK 


Attempts new things 
enthusiastically 


Exiiibits willingness to 
try new things 


Refuses to help others 


Volunteers to help 
others 
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VI. Work Pride 



4 2 



5 

Performs all aspects of 
work conscientiously 


3 

Does not volunteer for 
additional duties, but 
performs them v^hen 
asked 


1 

Abuses company- 
provided equipment or 
tools 


Produces above quality 
standards 


Produces to quality 
standards 


Performs below quality 
standards 


Volunteers to works over 
when behind 


Works overtime when 
asked, but does not 
volunteer 


Refuses to perform 
additional duties 


Demonstrates high self- 
esteem 




Demonstrates low 
initiative 
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VH. Making Decisions 



4 2 



5 

Judges work as good or 
bad without assistance 


3 

Occasslonally makes 
decisons on own 


1 

Walts until told before 
beginning anything 


Usually makes decisions 
on own 


Exhibits mature 
behavior 


Shows low concern 
about production or 
quality 


Uses own Judgement 
about when to start next 
job 


Starts jobs as they are 
assigned 


Exhibits immature 
behavior 




Asks for approval 
before starting next job 





Job Performance 
Skill Rating 



Vm. Leadership 



4 2 



5 

Expresses a desire to 
excel 


3 

Learns to work with 
others and how to 
instruct if necessary 


1 

GiveG poor instructions 


Thinks for himself and 
takes action to perform 
tasks or solve problems 


Gathers information 
from others to be able 
to begin a task 


Waits to be told to 
begin a task 


Trains quickly 


Demonstrates 
willingness to learn and 
asks questions 


Expresses no desire 
for improvement 


Takes initiative to ask 
questions to upgrade job 
knowledge 


Follows set procedures 
without error 


Work requires 
additional inspection 


Voluntarily helps others 




Blames others; 
demonstrates 
unwillingness to 
accept blame for own 
mistakes 


Requests additional 
training 




Constantly behind in 
work 


Sees ahead what's 
happening next 




Refuses help from 
others, even when 
needed 


Takes charge of assigned 

tcloxVo I CdUiiy 






Is seen to be trusted by 
coworkers 






Exhibits confidence in 
being able to perform job 
tasks well 







Job Performance 
Skill Rating 



IX. Taking Responsibility 



4 2 



5 

Demonstrates 
willingness to accept 
responsibility 


3 

Accepts responsibility 
when asked to 


1 

Exhibits or expresses 
unwillingness to 
accept responsibiltiy 


Volunteers for or 
requests opportunity to 
periorm new or 
additional tasks 


Corrects processes as 
instructed 


Blames others for 
mistakes 


Looks beyond problems 
for causes 




Walts to be told or 
shown what to do 


Looks ahead to suggest 
improvements 
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X. Resolving Conflict 



4 2 



5 

Willing to listen to 
others' views 


3 

Makes suggestions for 
resolving conflict when 
asked 


1 

Keeps trouble stirred 
up; keeps team 
disrupted 


Is seen as having respect 
of coworkers 


JViaKes suggestions 
about the causes of a 
conflict 


constant unhappiness 
with his job 


to arbitrate in conflicts 




Demonstrates inabilitv 
to stay focussed on job 
details; becomes 
emotional 






Demonstrates 
unwillingness to listen 
to views of others 
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NATIONAL WORKPLACE LITERACY PROGRAM 

T.O.P, PROGRAM 
(TRAINING OPPORTUNITIES PARTOERSHIP) 

S-rUDENT RECORD SHEET 

Naitve: 

Class: GED Basic Skills 

COMPANY: 

T.A^B.E, Scores: Reading Math 

Pre-Test Scores (Fill in scores or N/A if not tested in that subject area) 

Math Pre-Test with level assignment written on test 

Spelling Pre-Test 

Reading Pre-Test 

Post-Test Scores 

Math Post-Test 

Spelling Mastery Tests 



Spelling Post-Test 

Reading Post-Test 

Pre-GED Test 

Attendance: No. of classes held 
No» of absences 

Coiratvents or anecdotes: 



Each student folder should contain: 
Admissions Form (your option) 

Copies of any of the above tests administered to the student 
(I have TABE test copies) 
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NATIONAL WORKPLACE LITERAGf PROGRAM 

T,0,P. PROGRAM 
(TRAINING OPPORTUNITIES PARlMERSHIP) 

SKILL BATING SHEET 



NAME : PLANT ^ 

Please indicate your assessment of each student at program entry in the skill 
areas shown. Circle the number that best describes the student's level of 
functioning as he/she entered class. 

The chart describes the characteristics exhibited at levels 1, 3, and 5. Levels 
2 and 4 would be used for those who do not fit "neatly" into one of the three 
descriptions. 

We will use this same rating sheet at the end of classes in May. The two 
assessments will then be coiqpared to determine individual changes in skill 
level . 

After completing this for each student on your roll, please return to me in the 
enclosed envelope by December 18. 

I. SELF-ESTEEM AND PEE^SONAL GRCWTH 

5 4 3 2 1 

I. SELF-ESIEEH AND PERSONAL GROWIH 



5 

Demonstrates good 
attendance 


3 

Attends only 50% of the 
time 


1 

Has poor attendance habits 


Has desire to learn 


Is beginning to enjoy 
learning and will seek 
help 


Reluctant to ask for help 


Willing to share 
ac compl i shment s 


Willing to try things on 
his own 


Is afraid to try 


Shows confidence in 
skills and self 


Exhibits growing 
confidence as he enjoys 
success 


Is embarrassed over skills 


Has a positive 
attitude and outlook 
on life 


Has an improved attitude 
and outlook on life 


Has a negative attitude 
toward learning and life 


Has an improved 
physical appearance 
and demeanor 


Shows a somewhat improved 
appearance and demeanor 


Has a very 
self-depreciating 
appearance and demeanor - 
hunched^ quiets withdrawn 
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NATIONAL WORKPLACE LITERACY PROGRAM 

T.O.P. PROGRAM 
(TRAINING OPPORTUNITIES PARTNERSHIP) 

SKILL RATING SHEET 



NAME : . 

Please indicate your assessment of each student at program entry in the skill 
areas shown. Circle the number that best describes the student's level of 
functioning as he/she entered class. 

The chart describes the characteristics exhibited at levels 1, 3, and 5. Levels 
2 and 4 would be used for those who do not fit "neatly" into one of the three 
descriptions. 

We will use this same rating sheet at the end of classes in May. The two 
assessments will then be compared to determine individual changes in skill 



level . 



After completing this for each student on your roll, please return to me in the 
enclosed envelope by December 18. 



II. MATH SKILLS 
5 



II. MATH SKILLS 



5 

Understands number 
relationships and 
concepts 


3 

Basic understanding 
of operations 


1 

Has poor understanding 
of number relationships 


Able to perform 
mathematical operations 


Often needs help in 
performing basic operations 


Cannot perform basic 
operations 


Able to apply 
mathematical reasoning 
skills to situational 
problems 


Beginning to relate 
mathematical reasoning 
to situational problems 
and to everyday life 


Cannot apply reasoning 
skills 


Has problem solving 
capability 


Beginning to understand 
word problems^ but may 
need some help 


Has fear of word problems 


Exhibits confidence in 
math skills 


Shows increased 
confidence as skills 
inprove 


Exhibits math anxiety 



NATICMVL WORKPLACE LITERACY PROGRAM 

T.O.P. PROGRAM 
(TRAINING OPPORTUNITIES PARTNERSHIP) 

SKILL RATING SHEET 

NAME !^ - 

Please indicate your assessment of each student at program entry in the skill 
areas shown. Circle the number that best describes the student's level of 
functioning as he/she entered class. 

The chart describes the characteristics exhibited at levels 1, 3, and 5. Levels 
2 and 4 would be used for those who do not fit "neatly" into one of the three 
descriptions. 

We will use this same rating sheet at the end of classes in May. The two 
assessments will then be compared to determine individual changes in skill 
level . 

After completing this for each student on your roll, please return to me in the 
enclosed envelope by December 18. 

Ill, READING AND LANGUAGE ARTS SKILLS 

5 4 3 2 1 

III. READING AND lANGURGE ARTS SKILLS 



5 

Likes to read 


3 

Willing to read and has 
success at his level 


1 

Avoids reading 


Comprehends what he 
reads 


Conqprehends some of what 
he reads 


Cannot read and follow 
directions 


Recalls details and 
retains information 


Demonstrates some ability 
to recall and retain 
information, but needs 
reinforcement of 
comprehension skills 


Has difficulty 
comprehending what has 
been read 


Able to discuss and 
share material read 


Improved comprehension 
and verbal ability 


Unable to discuss and 
share material read 


Has large vocabulary 


Shows observable increase 
in vocabulary 


Has poor vocabulary 


Able to use reference 
resources 


Needs help in using 
reference resources 


Unable to use reference 
resources 


Can successfully 
communicate by 
writing 


Shows improved thought 
pattern and has limited 
success in communicating 
in writing 


Cannot successfully 
communicate in 
writing 


Exhibits good spelling 
skills 


Has moderate or fair 
spelling skills 


Exhibits poof spelling 
skills 
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PROJECT SCHEDULE 



The accon^nying project timeline shows the tasks involved in project 
implementation and the anticipated schedule of corrpletion. All tasks were 
completed on schedule and all established objteCtives met. 

As noted previously^ additions were made to the program with the expansion from 
three to five industry sites. A further outgrowth of the program, vrtiich was 
approved by the Department, was a PRE-SPC MATH class offered by one site. This 
course was provided on-shift and was extremely successful in adding 70 students 
to the program. Further, it showed a nearly 95% coirpl'^^tion rate and an average 
pre-post test score improvement of 26 (pre-test) - 88 (post-test). 



PROJECT TIMELINE 



Person Project Quarters 

T^sks Responsible 1st 2nd 3rd 4th 5th 6th 

Project Setup/Staffing: Project Director 

Data collection and re- 
porting procedures 
established 



Formative review of project Project Director 
by third party evaluator External Evaluator 



Bulls-Eye Needs Assess- 
ment: Staff facilitators 
will perform needs assess- 
it^nt DACUM workshops with 
management and labor 
representatives from each 
partnership industry 



Project Director 
Counselors 
Partnership 
Industries 



Course Development: The Project Director 
director will contract with Course Developers 
course content experts to 
translate the DACUM chart 
of needs into an appropri- 
ate curriculum design 



Orientation Seminars will Project Director 
be presented at each part- Project Coordinator 
nership industry to acquaint 
Management , supe rvi so r s , 
and enployees with the 
training program 

Enrollment of Employees Project Director 

Project Coordinator 
Comselors 



Motivational Seminar will 
be offered to enrollees as 
a positive "kick-off" to 
participation 

Assessment of Employees: 
Employees will be tested to Counselors 
measure current skill level Partnership 
functioning; data will be Industries 
referred to counselors and 
student/counselor assign- 
ments made 



Project Director 



Project Director 
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Tasks 

Employees will meet with 
assigned counselor to re- 
view scores, to determine 
individual goals and to 
decide on placement within 
the training program 

Classrooms will be estab- 
lished at each partnership 
industry site and outfitted 
with books, appropriate 
seating, overhead projec- 
tion, chalk boards and 
other materials necessary 
to carry out training 

Schediiling and release tirne 
or compensation arrange- 
ments will be determined 
and implemented 

Instructors will be 
selected according to 
qualifications and type 
of training needed 

Training of e^iployees 
either on site or with 
individual tutoring 

Ongoing employee co\an- 
seling and evaluation 

Project review and 
monitoring 

Employee evaluation 



PerscHi 
RespcHisible 

Counselors 



Project Quarters 
1st 2nd 3rd 4th 5th 6th 



Project Coordinator 
Partnership 
Industries 



Project Director 
Partnership 
industries 



Project Director 



Instructors 
Tutors 



Project Coordinator 
Counselors 

Project Director 
External Evaluator 

Counselors/Instructors 
Partnership 
Industries 



Final Project Review 



Project Director 
External Evaluator 



Final Project Reporing 



Project Director 
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PROJECT PARTICIEANTS 



Tables are provided showing participant numbers and characteristics. 

Measurement of progress or program conpletion differed among the various types 
of courses being offered. For those students in the Basic Skills or GEO 
curriculiim, a certificate of completion was provided. Of the 321 registered, 
156 received certificates (49%). In the Pre^SPC Math, 67 certTFicaes were 
awarded, representing a 95% conpletion rate. AdditionaT^data relating to GED 
completion is foiond on a separate table. 

The progress of students involved with literacy tutors from the Greenville 
Literacy Association is measured by the number of instructional books conpleted. 
Of the 192 students in the tutoring component, 111 had progressed at least one 
book level from their point of entry into the program. 



NATKmL WORKPLACE LITERACY PROGRAM 
INFORMATIW FORM 
T.O.P. PROGRAM 
No. V198A10150 



Part l! Program Parameters 

1. Target No. to be Served: 450 

2. No. Served at Each Site to Date: 

Site 1: Orientation 342 Assessment 409 

Site 2: Orientation 239 Assessment 239 

Site 3: Orientation 197 Assessment 130 

Site 4: Orientation 436 Assessment 119 

Site 5: Orientation 1043 Assessment 237 

3. Total No. served: Orientation 2257 

Assessment 1134 
Enrollment 513* 



4. No. Participating in Programs Offered: 

Literacy Tutoring 192 
Basic Skills 172 
GED 149 
Child Care 3 
Transportation 0 
Mini Math Courses* 22 
Pre-SPC Math* 70 

5. Contact Hours Provided: 

32 hrs./per 8 week session 
(Contact hours are the nvanber of 
teaching hours that workers receive) 



Am. Indian/ 
Alaska Native 1 



Part 2: Participation Data 
1. Mean Age Participants: 41.1 

3. Face/Ethnicity: No. who are: 
White 332 
Black 176 

Asian/Pacific 
Hispanic 2 Islander 2 

4. Years with the coitpany 

Unenployed 
0-5 
6-10 
11-15 
16-over 



2. Sex: No. Males 217 No. Females 296 

(Data for Pre-SPC Math Provided on 
separate sheet) 



No. Participants 
0 

135 

133 
TS5 



*TOTAL enrollment with Mini-Math classes 605 
and Pre-SPC Math 
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DEICGRAPHIC PROFILES BY SITE 



NED 



PLANT 


AGE 


W 


B I 


A 


H 


M 


F 


HCMELITE 


43.6 


36 


28 






28 


36 


JPS SLATER 


40.2 


69 


14 






46 


37 


JPS DUNEAN 


44.4 


46 


29 






40 


35 


JPS PARKER 


37.8 


86 


51 1 






37 


101 


JPS MCmGHAN 


39.5 


95 


53 


2 


1 


65 


86 



HIGHEST QRADE CXYIPLETED 





8th 


9th 


10th 


11th 


12th 


HCMELITE 




3% 


4% 


8% 


18% 


JPS SLATER 


18% 


13% 


10% 


7% 


5% 


JPS DUNEAN 


9% 


6% 


12% 


3% 


4% 


JPS PARKER 


10% 


13% 


27% 


20% 


37% 


JPS mmCHMi 


24% 


33% 


19% 


9% 


12% 
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PRE-SPC MATH 
JPS -'PARKER 

DEMOGRAPHIC DATA 

RACE sax GRADE COMPLETED 
AVG AGE W B M F 2 2 ISI 11. 12 

38 38 32 9 61 2 6 10 9 40 

ATTENDANCE 

ENROLLED ATTENDANCE 

70 94% perfect attendance 

AVG. PRE-TEST AVG. POST-TEST 

26 88 



G.E.D. FINAL INFORMATION 



MGNAGHAN 



ENROLLED 49 

DROPPED 16 

RECOMMENDED FOR GED TEST 15 

RECEIVED GED 4 

ATTENDED 50% OR MORE 13 

SHOWED GRADE LEVEL IMPROVEMENT *7 



HOWELITE 

ENROLLED 11 

DROPPED 0 

RECCmENDED FOR GED TEST 1 

RECEIVED GED 1 

ATTENDED 50% OR MORE 5 

SHOWED GRADE LEVEL IMPROVEMENT 6 



PARKER 

ENROLLED 55 

DROPPED 31 

RECOMMENDED FOR GED TEST 10 

RECEIVED GED 2 

ATTENDED 50% OR MORE 6 

SHCWED GRADE LEVEL IMPROVEMET^ 7 
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PROJECT DISSEWINATICN 



The Training Opportunities Partnership was featured as a concurrent session 
topic at the American Association of Adult and Continuning Education Annual 
Conference in Annaheim, California in November 1992. The presentation, enitled 
•'BASICALLY, A Tough Road to Conquer" focused on the need for literacy education 
in the workplace and the methods used by the Training Opportunities Partnership 
to meet that need. 

To further share the experiences and knowledge gained by the project, a 
methodology report has been prepared and made available both through state and 
national channels. 
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EVALUATION ACTIVITIES 



Jorie Philippi was contracted as the external evaluator for the T-O.P. project* 
Under her direction, evaluation strategies were developed and impleinfented. 

These strategies dealt with the project's infract in 3 areas: skill level 
achievement, job productivity, and personel growth and development. Both an 
explanation of the nvethods utilized and an analysis of the evaluation data are 
included in the report sutmtted by Ms. Philippi. 
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Introduction 

Background : The Career Advancement Center for Greenville Technical College in 
Greenville, South Carolina was funded by an 18-month grant award from the \J,S, 
Depjirtment of Education to provide workplace literacy services to two area industries v/ith 
whom they partnered, namely, JPS Textiles and Homelite. The program was granted a 3- 
month no-cost extension and operated as a national workplace literacy project demonstration 
from March 1, 1991 through November 30, 1992 to deternnine tlie effectiveness of the 
Center's proposed workplace applications of basic skills training model. 

The need for this project grew from a recognition by local businesses and industries 
that the pressures of competition in a global marketplace have accelerated the pace of change 
in workplace environments. The expanding and shifting responsibilities of manufacturers 
and businesses in transition from Taylorism to a Total Quality Management system via self- 
directed cross-functional work teams, increasing technological demands and reduced 
PiOduction cycle times, along with the advent of national testing requirements for operators 
and drivers, have created an interest among employers and workers alike to enhance use of 
the workplace basic skills needed by the Greenville area labor force to meet these 
challenges. Because technical training-specific courses and traditional education often do 
not give workers a broad-based knowledge of team communication, problem-solving, 
critical thinking and leaming-how-to-leam concepts and competencies, participating 
companies detennined the need for instructional programs that would provide workers with 
workplace basic skills applications that are transferable and adaptable to their changing work 
environments. 

Area companies began discussions with Greenville Technical College in 1988, 
following the formation of the Governor's Task Force on Excellence and hiring of a 
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workforce specialist, assigned to the college. Initial outreach services to community 
businesses helped to foster the sharing of information and to clearly define company/ 
worker needs and agency responses. This careful exploration of possibilities resulted in the 
paiinering of four JPS Textile facilities and a Homelite plant to apply for federal grant 
monies for provision of on-site programs. (Initially, Cryovac, another area manufacturing 
plant, had agreed to be a partner as well, but terminated their participation due to 
circumstances external to the project.) Managers represendng the organizations' training 
and education departments met with the college program developers to ensure that the 
customized programs directly related to the competencies needed for the workplace and 
responded to the needs of the targeted worker piu'ticipants. To this end, the 
education/business partnership members were committed to gathering data for perfoiraing a 
"front-end analysis" in order to assess the communication and problem-solving needs of 
targeted worker-participants. They also determined programs* goals, lengths, schedules, 
and implementation plans. 

The developers of the programs, members of the staff at the Career Advancement 
Center at Greenville Technical College, then custom-designed, created and delivered the 
instructional program. A variety of strong programs were subsequently implemented and 
refined during the grant period. Greenville Technical College, as the grant financial 
manager, contracted with Performance Plus Learning Consultants, Inc. to serve as a third- 
party evaluator throughout the project. 



Purpose of the Evaluation : The Career Advancement Center at Greenville Technical College 
has requested this third-party evaluation of their U.S. Department of Education Workplace 
Literacy Demonstration Project to assess 1,), the extent to which the project's goals and 

Prepared by Performance Plus Learning Consultants, Inc. 

ERIC 5G 



Greenuffle, SC TraJi-\}ng Oppatunmes Parlxiership 

DOE Workplace LUeracy Grant Program- Final Evaluation Report 



5 



objectives have been accomplished, and 2.), tlie extent to which program development and 
implementation proceeded as planned. Stated program goals were: 

1 . ) to improve the productivity of the labo/ force through improvement of basic literacy 

skills needed in the workplace; and 

2. ) to enhance workplace literacy by collaborating with industry to provide the resources 

necessary for literacy training in the workplace, reach employees who otherwise 
would not have access to literacy programs, and increase public awareness through 
an effective media campaign that will emphasize the severity of the problem of 
adult literacy. 

Specifically, the evaluation objectives to be investigated were: 

on-going identification of the program's strengths and areas still needing 
any improvement throughout the life of the project; 

evidence of a reduction in the incidence of workplace Uteracy in Greenville 
County through the establishment of workplace literacy partnership 
programs at three area companies and assessment of at least 200 employees 
of each company; 

evidence of the development of a training methodology for adult learners that 
meets specific workplace needs, with individualized materials for each 
industry that utilize that indusL y's available job-related materials; 

evidence that the project progr.ins implemented targeted and delivered 
services to a minimum of 150 workers at each industry location and using 

Prepared by Performance Plus Learning ConsulLanis, Inc. 
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initial assessment score's to identify eligible program participants; 

evidence of a smooth instructional flow of activities witliin the developed 
cuiTicula. reflecting a sound developmental approach to mastering those 
literacy skills necessary for competent perfomiance of identified job tasks; 

• evidence of the offering of additional services to each project participant 
through the use of an individualized program of literacy instruction, 
supported by personal counseling, tutorial services, cliild care, and/or 
transportation assistance, and subsequent course completion by 60% of 
participants or more. 

• evidence that measurement of the impact of training on project participa^us 
was conducted via internal evaluation processes that tracked 1, literacy skill 
achievement by means of pre- and post-test scores; 2, job productivity; and 
3, personal growth and development; 



evidence of successful program implementation tlirough the use of 
appropriate processes for participant recruitment, class scheduling, 
development of individual education plans, cumculum delivery, pre- and 
post-assessment, and instructor training and support, that aie acadeixncally 
and organizationally sound and that match with program goals; and, 

evidence of a county-wide awareness campaign about adult literacy 
conducted through various local media and of a final project report on 
methodology and results available for national dissemination. 
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Additionally, recomniendations were requested on the issues and concerns about model 
replicability, limited to data gathered from observations, survey and test results, anecdotal 
records and interview information. 



Description of the Project to he F,vah.?^ ted: The Greenville Training Opportunities 
Pannership (TOP) Workplace Literacy Project consisted of five workplace literacy training 
partnerships fonmed between Greenville Technical College Career Advancement Center and 
the plants of two local industries. The demonstration project partners 



were: 



Homelite, a division of Textron, Inc., which manufactures chain saw 
components, construction pumps, and construction generators and employs 
approximately 430 workers. The TOP program operated at this site from 
September, 1991 through May, 1992. Twenty-two employees enrolled in the 
upgrading strand, four dropped out, and 14 received certificates of 
participation. Eleven enrolled in the GED preparation strand, none dropped 
out, and one received a GED. 

JPS Textile Group, Monaghan Plant, which manufactures unfinished cloth 
and employs approximately 740 workers. The TOP program operated at this 
site from August, 1991 tlirough May, 1992. Thirty-two employees enrolled in 
the upgrading strand, nineteen dropped out, and thirteen received certificates 
of participation. Forty-nine enrolled in the GED preparation strand, sixteen 
dropped out, and four received GEDs. 
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JPS Textile Group, Parker Plant, which manufactures automotive products. 
The TOP program operated at this site from October, 1991 through May, 
1992. Fifty-four employees enrolled in the upgrading strand, twenty-two 
dropped out, and thirty-two received certificates of participation. In 
addition, three Pre-SPC math courses were conducted on company time with 
a total enrollment of ninety-five participants; average attendance ranged 
from 88% to 100%. Fifty-five employees enrolled in the GED preparation 
strand, thirty-one dropped out, and two received GEDs. 



JPS Textile Group, Slater Plant, which manufactures fiberglass cloth. The 
TOP program operated at this site from September, 1991 through May, 1992. 
Fifty-two employees enrolled in the upgrading strand, twenty-eight dropped 
out, and twenty-two received certificates of participation. 

JPS Textile Group, Dunean Plant, which manufactures unfinished clotli. The 
TOP program operated at this site from August, 1991 through May, 1992. 
Thirty-four employees enrolled, five dropped out, and 30 received 
certificates of participation. 



was 



According to the published description of the program, the design of the project 
structured to meet workers' basic skills needs through the development and/or provision of 
academic and job-related curricula. DACUM-style job analyses conducted by technical 
college staff resulted in the development of curricula and instructional delivery formats 
tailored to meet the various employer/worker needs of each of tlie partners. A brief 
description of the program follows: 
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Three strands of instruction were made available to individual participants: non- 
reader tutoring, GED preparation, and basic skills upgrading (job-related). 
Participants were placed in a strand through analyses of their scores on pre-program 
assessment instruments and through intake interviews with counselors and 
instructors. Each participant attended on a voluntary, non-reimbursable basis during 
off-work hours. Classes at each site were conducted before and after shifts and used 
an open-entry, open-exit format. In addition, several pre-SPC math classes were 
requested, developed, and operated toward the end of the grant period at several 
plant locations. 

Counselors visited each site on a regular basis to meet with participants. They also 
placed follow-up phone calls to absentees to determine reasons and offer 
interventions, where possible. Child care and transportation were made available io 
all participants, although few opted to utilize these services. 

Curriculum materials for non-reader tutoring were provided on an individual basis 
by the Greenville Literacy Association. GED preparation was prescribed by 
instructors at each site and utilized commercially available texts and workbooks. 
The basic skills upgrading curriculum materials were developed by the Greenville 
Career Advancement Center, based on the results of two DACUM job analyses 
conducted with separate panels of managers and employees from Homelite and from 
the four JPS locations, using the Bulls Eye technique. The customized materials 
tailored to the needs of each of the two companies consisted of: 

- TOP Reading for Homelite - a loose-leaf binder of 1 1 8 pages, consisting of 
ten units of instruction: 
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I. Getting an Overview of Materials 

II. Using an Index 

HI. Following Directions 

IV. Using Context Clues 

V. Locating the Main Idea 

VI. Discovering Details 

VII. Reading Between the Lines 

VIII. Reading Charts, Tables and Graphs 

IX. Putting Ideas in Order 

X. Developing Scanning Skills 



- TOP Reading for JPS Industries^ a loose-leaf binder of 126 pages, 
consisting of ten units of instruction: 

I. Getting an Overview of Materials 

II. Using an Index 

III. Following Directions 

IV. Using Context Clues 

V. Locating the Main Idea 

VI. Discovering Details 

VII. Reading Between the Lines 
VIII. Reading Charts, Tables and Graphs 

IX. Putting Ideas in Order 

X. Developing Scanning Skills 

Both reading courses share core instructional materials for each unit, which focus 
materials and situations from everyday life, such as newspapers and recipes, to teach 
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the skills. Most units also contain one example of a company-specific job print 
material that required using an applied version of the skill(s). Paper-pencil exercises 
are the major activities for each unit, with minimal explanation, which is generaUy 
given in the form of rationale, rather than a modeling, or demonstration, of how to 
perf-orm the skill. Many examples and graphics appear to have been excerpted from 
other commercial materials, but citations jire not provided. 

TOP Math for Homelite- a loose-leaf binder of 62 pages, containing 16 units 
of instruction: 

Lesson 1.- Place Value, Rounding Numbers 

Lesson 2.- Addition and Subtraction of Whole Numbers 

Lesson 3.- Multiplication and Division of Whole Numbers 

Lesson 4.- Introduction to Fractions; Comparison of Fractions 

Lesson 5.- Fraction Addition 

Lesson 6.- Fraction Subtraction 

Lesson 7.- Fraction Multiplication 

Lesson 8.- Fraction Division 

Lesson 9.- Introduction to Decimals 

Lesson 10.- Addition and Subtraction of Decimals 

Lesson 1 1 .- Multiplication and Division of Decimals 

Lesson 1 2.- Averaging and Tolerances 

Lesson 13.- Ratio and Proponion 

Lesson 14.- Introduction to Percent 

Lesson 15.- Three Types of Percent Problems 

Lesson 16.- Basic Measurement 
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TOP Math for JPS Industries- a loose-leaf binder of 56 pages, containing 16 units 



of instruction: 




Lesson 


L- 


Place Value, Rounding Numbers 


Lesson 


2.- 


Addition and Subtraction of Whole Numbers 


Lesson 


3.- 


Multiplication and Division of Whole Numbers 


Lesson 


4.- 


Introduction to Fractions; Comparison of Fractions 


Lesson 


5.- 


Fraction Addition 


Lesson 


6.- 


Fraction Subtraction 


Lesson 


7.- 


Fraction Multiplication 


Lesson 


8.- 


Fraction Division 


Lesson 


9.- 


inuOQUCUon lO uecimais 


Lesson 


10.- 


Addition and Subtraction of Decimals 




1 1 

i 1 .- 


iviuiupiicduon lUio ui vision ui i-'ecinidio 


Lesson 


12.- 


Averaging and Tolerances 


Lesson 


13.- 


Ratio and Proportion 


Lesson 


14.- 


Introduction to Percent 


Lesson 


15.- 


Three Types of Percent Problems 


Lesson 


16.- 


Basic Measurement 



Both courses are designed to be used in conjunction with Contemporary Books 
mathematics series texts, Number Power I and Number Power 2. Each lesson 
consists of single- and multiple-step word problems, that incorporate workplace 
references, but do not relate to actual on-the-job application of quantitative 
techniques. For example, a typical word problem reads: 

" In a weave room, there are 645 looms. There are two harnesses on each 
loom, and each harness contains 890 heddles. How many heddles are there 
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in all the looms?" 

This does not appeal* to be the type of problem that would be encountered regulaily 
on the shop floor, requiring a worker to do multiple-step multiplication functions in 
order to perform an actual job task. 

Get Comfortable with Math, Parts I and II for J PS Industries- 
Approximately 100 loose pages of exercises that address whole number operations, 
fractions, decimals and percents. The materials, although bearing a cover sheet 
indicating their development by TOP project staff, contain references to a 
commercial text. Math that Works! and the pages appear to be copies of this book. 
No citation is provided. 

Industrial Math for SPC- A paper-bound 32-page text, developed by tlie 
Greenville Technical College Governor's Workforce Initiative, addresses the 
following topics: 

- Vocabulary used in SPC 

- Rounding 

- Adding Whole Numbers and Decimals; Averaging 

- Subtracting Decimals and Finding Range 

- Multiplying Decimals 

- The Metric System; Measuring 

- Understanding and Making Graphs 

Everyday life and generic SPC excmiples jire used to illustrate tlie concepts taught. 

Spelling for JPS Industries- A series loose-sheet modules of 1 to 10 pages 
each that provide instruction in: 
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- Spelling Skills 

- Using a Dictionary 

- Recognizing Prefixes, Suffixes and Base Words 

- A Brief Grammar Review 

- Words that are Frequently Mispronounced 
~ Plurals with -s and -es 

" Homophones 

- The One Hundred Spelling Demons 

- Put i Before e, Except after c 

- Mastery Test: Spelling and the Dictionary 

Modules contained both original materials and copies of pages from commercial 
materials. No citations were provided. 

Pretests and posttests were not contained in the curriculum materials examined 
Project staff reported scores from the Test of Adult Basic Education (TABE) in 
reading and math, as well as other pre- and post-test scores. There were no 
instructor guides developed to accompany the customized curricula {uid instructor 
training was reported to be minimal. 



ERLC 



Method 

Design : The evaluation of the Greenville TOP Workplace Litemcy Demonstration Project 
employed a modified version of the Context-input-Process-Product (C.I.P.P.) model, 
(Stufflebeam & Guba, 1971). This method of evaluation was chosen by the evaluator as the 
most suitable tool for investigating the evaluation objectives, (see pages 5-7), because it 
Prepared by Performance Plus Learning Consultants, Inc. 
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examines project effectiveness through structured analysis of tlie cohesivei*ess of project 
goals, components, and operations, independent from comparisons to outside standards or 
other programs. 

The C.I.P.P. model was used to analyze: 

Context: (i.e., shared goals and philosophy of key personnel and 
participants); 

IflCUt (i-e., resources, including personnel, materials, time and 
facilities); 

Pi2££Ss (i.e., congruence of obserx'ed instructional development and 
delivery with project goals and research on instructional 
effectiveness); and 

Ptoduct (i.e., indicators of project effectiveness). 

It is important to note that, due to the number of partnerships and limited resources 
allocated to evaluation in tiiis project, extensive and uniform investigation at all sites was not 
possible. Fomis and procedures for use in data collection across sites were developed by 
PPLC and explained to representatives and staff for each partnership, as well as to the 
project director. 
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Participants: The participants in the project were workers employed by the partner 
companies. A brief description of the available composite average worker profiles by sites 
is provided below for reference. 



Demographic Characteristics by Site 


Flam 


Median Age 


Male 


Female 




Honielite (n= 64) 


43.6 years 




44% 


56% 




Jro oiaier [u— oj) 


40.2 years 




55% 


45% 




JPS Dunean (n= 75) 


44.4 years 




53% 


47% 




JPS Parker (n= 138) 


37.8 




27% 


73% 




JPS Monaghan (n= 151) 


39.5 




43% 


57% 




Ciant 


Ethnic Groups: 










White 


eiack 


Indian 


Hispanic 




Homelite 


56% 


43% 


0% 


0% 


0% 


JPS Slater 


83% 


17% 


0% 


0% 


0% 


JPS Dunean 


61% 


39% 


0% 


0% 


0% 


JPS Parker 


62% 


40% 


8% 


0% 


0% 


JPS Monaghan 


63% 


35% 


0% 


1% 


1% 


Plant 


Highest Grade Comoleted in School: 






8th 




IMli 


iUli 


mil 


Homelite 


0% 


3% 


4% 


8% 


18% 


JPS Slater 


18% 


13% 


10% 


7% 


5% 


JPS Dunean 


9% 


6% 


12% 


3% 


4% 


JPS Parker 


10% 


13% 


27% 


20% 


37% 


JPS Monaghan 


24% 


33% 


19% 


9% 


12% 



Because of the nature of the evaluation design, the focus of evaluation activities 
extended beyond the traditionally-held concept of "participants" to also include project 
administrators, the employers, instructors, counselors, and developers. 



Instruments : Data for this evaluation were requested and gathered via pre- and post- 
program learner surveys, structured interviews with learners and program personnel, 
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instructor anecdotal report forms and questionnaires, supervisor ratings, and formally- 
documented observations of instructional sessions and instructor training. Additionally, 
data were gathered from detailed analysis by the evaluator of program documentation, 
instructional materials, and leai*ners' work, ( i.e., pre- and posttest scores and learners' 
records). 

Procedure : Following initial telephone conversations with key personnel at Greenville 
Technical College Career Advancement Center to establish evaluation objecdves, the 
evaluator conducted the activities listed below. On-site visits were conducted during June 
and October, 1991 and June, 1992. 

1 . Development of Evaluation Data Collection Instruments: 

• Fonns created for Participant Pre- and Post-Program Surveys, 
Instructor Interview, Instructor Anecdotal Records, Learner 
Individual or Focus Group Interview, Classroom Observation, 
Employer/Supervisor Interview and Supervisor Performance 
Indicator Ratings. 

2. On-site consultation with curriculum developers concerning instructional 
curricula design and development and feedback on how to strengthen 
activities contained in them. 

3. On-site interviews with training and production managers, supervisors, 
instructors, project managers and learners from the five partnering industry sites. 
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4. On-site observations of classes during various cycles of instruction. 

5 . Off-site analysis and review of materials and collected data from sites. 

6. Communications and Operations: 

• Contact throughout grant period with project through conversations 
with project director Joan Mason, to discuss project goals, 
progress, and evaluation activities. 

• In-person interviews with instructors, curriculum developers, 
and counselors. 

• Final Evaluation Report submitted to project director, February, 
1993. 

Results 

Project Context : 

To what extent are goals and philosophy of the project shared by key 
project personnel and learners ? 

This section of the evaluation is a comparison of the project goals and priorities as 
reported in project descriptions and interviews with key project personnel, including: 
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• project diiector 

• project curriculum developers; 

• managers and supervisors; 

• project instructors; 

• project counselors; and 

• learners. 

These viewpoints about project goals were analyzed for consensus and divergence. 

The published project goals and purposes are contained in the grant proposal 
submitted to the U.S. Department of Education. They \/eie developed cooperatively after 
almost one year of meetings and communication between Greenville Technical College and 
partnering industries, prior to applying for the grant monies. Stated goals and objectives in 
the grant proposal included: 

to improve the productivity of the labor force through improvement of basic 
literacy skills needed in the workplace; 

to provide the resources necessary for literacy training in the workplace; 
to reach employees who otherwise would not have access to literacy 
programs; 

to increase public awareness through an effective media campaign that 

empliasizes the severity of the problem of adult illiteracy; 

to assess and provide instruction to at least 600 employees in the partnering 

companies; 

to develop a training methodology for meeting the basic skills needs of adult 
workers that can be disseminated nationwide; 
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♦ to offer ancillary services for child care, transportation, and counseling to 
every participant in order to encourage better program attendance and reduce 
drop-out rates; 

♦ to increase the workplace literacy levels of at least 450 program participants; 

♦ to increase the grade level abilities of each participant by one grade level per 
session attended; and 

♦ to develop a manual that details project procedures and techniques for use by 
other business/education partnerships operating similar programs. 

Project Director - On June 10, 1991 Joan Mason, of Greenville Technical College 
Career Advancement Center, was interviewed about her perceptions of program goals and 
philosophy. Speaking for herself and the Career Advancement Center, she articulated the 
following project goals: 

♦ to increase the basic skills of the workforce to the point of meeting 
workplace demands, i.e., more of the technical on-the-job requirements 
for getting the job done. 

to create materials that ai'c at least 75% specific to the job. 

♦ to develop a program that results in participants who are retained and don't 
drop out. 

Project Curriculum Developer - Linda Garner was interviewed during tlie June, 1991 
site visit . The goals expressed centered around developing curriculum that enabled workers 
to be trained by their companies to do the jobs required of them. In addition, an emphasis 
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was placed on the need for the curriculum to be 50% work-related and 50% non-work 
related, to ensure that participants would find multiple uses for what they learned. The other 
two curriculum writers, Corrie Wiley and Susan Bramlette, were also interviewed during 
the visit and expressed similar opinions. 

Managers and supervisors - were interviewed at each of the partner plant sites during 
the June, 1991 site visit. Comments about project goals generally included references to 
improving the general academic and everyday life basic skills of workers, with littie 
evidence of a desire to connect course content to job contexts or companies' goals. 
Structured interviews were conducted with personnel managers Bob Fletcher of JPS Slater, 
Ronnie Brown of JPS Monaghan, Jyles Phillips of Dunean, David Brown cf JPS Parker, 
and human resources manager Ed Hindman of Homelite in Greer. Responses to the 
question. What do you consider to be the main purposes, goals and objectives of this 
program? included: 

"The goal is not necessarily work-related; rather, we want to provide our workers 
with an opportunity to get education that will impact on their lives. If they [the 
workers] can improve their quality of life, their personal goals can be accomplished, 
like helping their kids with homework." 

"Employees who are parents of children need to change their own attitudes about 
education, an intergenerational approach." 

"We want to help our employees pursue areas of education they didn't get to explore 
in school- for whateve/ reasons- and make school convenient for them now." 
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*'If they [the workers] improve in their everyday life skills, maybe they'll be able to 
improve their ability to handle new equipment in the future." 

"If they [the workers] learn to read, they can read anything; two times two is the 
same in their checkbooks as it is on the job. Hopefully, they'll achieve a better 
quality of life than they have now; n^^riybe that will help them become more 
promotable." 

"We want to improve and upgrade the basic skills of our workforce to enable them 
to build a better self-image. We'd like to see them [the workers] expand their ability 
to enjoy life with things like reading, to travel, and so on. If they can use these 
same skills on the job, they'll be better motivated to work." 

Only two of the managers interviewed. Brown and Hindman, said that they thought 
they might be able to identify any workplace indicators that could be used to measure the 
transfer of learning and impact of the program on worker performance of job tasks. When 
asked how they would know if program goals were being met, responses included: 

"When I hear [worker] comments like, 'I helped by grandbaby do math homework.* 

or Tm really enjoying this class.' That's success. We'll be accomplishing what 

we set out to accomplish." 

"The only way we can really know is to test at the end of the program and see if they 
learned the reading and math that was taught. That way, the participants will have 
a sense of accomplishment in life." 
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"I'll know when I hear an employee proudly say, 1 just completed Book 1/ or 
when I attend those graduation ceremonies next year for the GED recipients." 

"We'll want to know if the participants are reaching their academic objectives, that 
is, how many get GEDs or finish their studies. Their [the workers'] attendance rates 
in class, retention, and attention during class also will tell us how well the program 
is going." 

Only one manager mentioned positive feedback from shopfloor supervisors as an 
indicator of achieving program goals. 

The evaluator also conducted focus groups with supervisors at each plant location to 
elicit attributes of competen^ average, and less than competent workers in order to develop 
an instrument for benchmarking and measuring program impact by means of identification 
and employee ratings with local performance indicators for each site. After reviewing the 
results of all the focus groups, the supervisors at the separate locations opted to combine 
their responses and use one project-wide perfomiance indicator instrument for rating of all 
project participants. The performance attributes selected for inclusion in the employee rating 
instrument were: 

- dependability 

- working as a team member 

- self-initiative 

- job knowledge 

- adaptability 

- work pride 

- making decisions 
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- leadersliip 

- taking responsibility 

- resolving conflict 

Supervisors identified three sets of specific observable, measurable behaviors for 
each of the above attributes that would be exhibited by a highly competent worker, by an 
average worker, and by a less-than-competent worker. The results were used by PPLC to 
create an instrument that was subsequently used by the supervisors to conduct pre- and 
postprogram ratings of participants they supervise. 

Project Instructors - Instructors were interviewed separately in their classrooms 
during the site visits. Conraients indicated a desire to assist workers in mastering "real-life 
problems" and in "building their self-esteem." Many mentioned helping employees "feel 
comfortable" with the learning process. Two instructors felt that program emphasis should 
be on educating the whole person and building everyday living skills, rather than on work- 
related skills. Only one instructor voiced an opinion that the programs' content should 
address at least 50% work-related skills. All instnictors stressed their preference for using 
the commercial basic skills materials and the academir portions and examples from the 
customized curricula. 

When asked what the most important things for an instructor in tlieir program to do 
were, typical responses included, "to build rapport with the learners so they feel comfortable 
asking for help," "to be interested in learners academically and personally," and **to get 
familiar with the materials taught." When asked, How do you know when you're achieving 
the goals of the program?, instructor responses included: 
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" When people keep coming." 

"When someone says, 1 went to the store yesterday and was able to figure out 

percent for the first time.' or 'I never thought I could learn that.'" 

"When I see evidence that self-esteem is being built." 

"There are subtie ways of seeing the goals- I'm not sure I can explain it." 

"I look at the amount of work the learners are completing, that is, the dates on the 

packets they master. They [the workers] should be completing packets regularly and 

turning them in. I also can tell the program is going well if participants ask lots of 

questions; that indicates a high level of comfort with me and with the program." 

Learners - Goals of participants were collected in stmctured focus group interviews 
during site visits and on pre-program and post-program surveys administered by project 
staff throughout instructional cycles. Responses from participants in the program indicated 
the importance of having an opportunity to study skills they had forgotten or never 
encountered. Many mentioned math lessons as those that really met their needs. Specific 
goals included preparation for entering college or taking advanced technical courses or 
getting GEDs. All participants in the focus groups indicated wanting to acquire or improve 
their job-related basic skills. Responses evidenced that most participants wanted to either 
upgrade current job skills or prepare for future job/ career tasks. There was a general 
concern by focus group program participants conducted at JPS sites over the technological 
changes in the textile industry and their need to be prepared to change with job tasks. Many 
examples were provided of particular job applications in which they hoped to use what they 
were learning. 

Pre-program participant surveys contained an open-ended leanier goal statement 
stem. Participant responses indicated their expectations were as follows: 
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want my GED 

I would probably be the first to be laid off if I don't get this training 
need to learn to help myself and my highschool daughter 
I've forgotten things I knew in school. 

I need to know about fractions, percents and decimals; I'm gomg to conquer them. 
I want to learn! 

I took a correspondence course to finish high school and I feel I missed a lot. 
to increase my knowledge of math 
I'm here just because it's a challenge. 

I have a grandchild who will someday need help with school work and I want to 

be able to help. 
I quit the tenth grade. This is an opportunity, 
to finish algebra 

I want to upgrade enough to go on to Greenville Tech. 
This can help me learn more. 
I want a GED and better jobs, 
to get a better job. 
to be a tutor 

to help my kids a little more 
to have the job advantage, 
can't do math; need to leam 
to leam math for the future 
to be able to get a better job 
to go into horticulture 
better job opportunities 

The more I learn, the more I'll know about myself. 
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PPLC collected and analyzed goal statements from project director, project 
curriculum developers, managers, supervisors, project instructors, and participants. For a 
discussion of areas of convergence and divergence, please see the evaluation section, 
"Summary of Results," under Discussion . PPLC next investigated the input of resources to 
the project, which is addressed in the next section of the evaluation. 
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Project Input : 

What reso urces were available to the project during 
develo pment and implementation and to what extent 
were they yis ^d effectively ? 

This section of the evaluation addresses major resources of tlie project. It includes 
program instructional materials, design and appropriateness for the targeted learner 
populations; key personnel qualifications and the match between published project duties; 
and facilities. It also examines the content and processes used for instructor training. The 
data presented in this section were analyzed for strengths and weaknesses. 

Prograrn. materials- The instructional materials were designed for each of the sites 
after developers conductedpACUM job analysis, using the Bulls Eye technique with 
employee and manager focus groups from JPS and Homelite. Thirteen panelists from 
across all four JPS plant locations and fourteen panelists from Homelite participated in the 
processes as two separate groups. No information was made available concerning the 
numbers of managers versus employee panelists for each company. The panel group 
processes were conducted by Joan Mason, project director, and Al Stokes, a project staff 
counselor. 

Documentation of the DACUM analyses identified specific job task priorities and 
lengthy lists of critical print job materials for eacli of the two companies; however, other 
than use as occasional examples contained in the custorrilzed lessons, there is uv) evidence 
that formal curricula designs or contents were developed from the resulis of the DACUM 
job analysis to address company-identified job task basic skills application needs. The 
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similarity of instructional unit titles and content for both companies provides evidence to the 
contrary. 

The ranges of reading difficulty level for the various instructional materials appeared 
to match the ability levels of targeted participants. Results of pretests were the only 
measures of targeted participant abiHty levels. Customized pre-tests and posttests to 
measure learner mastery of the customized materials were not developed. Observations of 
delivery and analysis of classes demonstrated the ability of learners lo comprehend topics 
and to participate in learning activities comfortably. 

When asked about the strengths and weaknesses of instructional materials, the 
majority of participants thought the content reintorced skills needed on the job or in 
everyday life, especially in the conraiercially developed math materials. When asked to 
make recommendations for improving the program, over half of the participant responses 
addressed the curriculum materials. Recommendations included: 

- more math like I signed up for 

- more examples of how problems are done 

- more English skills 

- the reading took too much time 

- more class [whole group] work 

- get rid of the blue [customized] reading book; use packets instead 

- change the reading book; make ii less confusing 

- change the reading part of the program 

- add more time on computers 



Prepared by Performance Plus Learning Consultants, Inc. 



Greenville, SC Training Opportunities Partnership 

DOB Workplace Literacy Grant Program- Final Evaluation Report 



Instructor guidelines scripted for customized course sessions were not developed; 
rather, the materials consisted of only worksheets and handouts for participant use. 

K ^y Personnel - Program coordinators from the colleges had established working 
relationships with their business/labor partners prior to receipt of the grant. The curriculum 
writers had previous instructional writing experience and the co-developer of the SPC Prep 
course also had previous experience with development of functional contextual materials for 
workplace literacy programs. Instructors were seasoned adult basic education teachers with 
expertise and many years of teaching experience. Two hold masters degrees and two 
bachelors degrees. Major areas of study include chemistry, mathematics, social studies, and 
educadon. 

The project director, Joan Mason, holds a masters degree in social sciences and has 
had extensive experience in program management. The three counselors all hold masters 
degrees in guidance and counseling or personal services and have had extensive experience 
in working with adult clients. The curriculum developers and program supervisor Linda 
Garner, who worked with them, hold bachelors and masters degrees in education. 

Facilities Sessions were conducted in a vmety of worksite locations. The majority 
were held in training and meeting rooms of the panner companies. At two JPS sites, room 
was provided in an adjacent church and store front. All facilities observed during the three 
site visits appeared well lit and conveniently located, with adequate space for conducting 
learning activities. 

Instructor Training - Initial materials overview was provided for project staff Linda 
Garner to familiarize the instructors with the materials. Responses to interview questions 
indicated that instructors felt that their previous teaching experience had seasoned them to 
e^y^cmA thcfletfand^t'orfaHSqff a Greenville Technical 
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College representative with her in the classroom during the first session to answer questions 
from participants about the program. All suggested that having time to meet as a group of 
instructors with the rest of the project staff about half way through the first cycle would 
have lent support to their efforts, allowed them to give feedback to the developers, and built 
a better sense of shared learning community. 

For a discussion of strengths and weaknesses of available project resources and the 
effectiveness of their use, see "Sunamary of Results" under Discussion section of the 
evaluation. Tlie next section of this evaluation examines the process of project delivery. 
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Project Process : 

To what extent were program development and observed 
instruction congruent with project goals and research on 
instructional effectiveness ? 

Instructional Organization - Across sites, the typical class sessions held during the 
project met for 32 hours per 8-week cycle. The GED prep program strand was 
individualized. The customized basic skills upgrading strand instruction was conducted as 
group classes, as were the shorter mini-math courses held at several of the plants toward the 
end of the grant funding cycle. 

The nature of instruction and types of leeu-ning acuvities were determined through 
observation as well as interviews with both instructors and learners. Both learners and 
instructors reported that approximately 90% of instmctional time was spent working 
independently, in small groups or pairs, with less than 10% of time spent working as a 
whole group. Records from 3 class observations by the evaluator indicated an average of 
11% of instructional time spent in whole group instruction employing lecture techniques. 
This compares favorably with an ideal of less than 50% teacher-talk during any one 
instructional session (Goodlad). On-site interviews and observations occurred one time 
during the middle phase of tlie project, in October, 1991. 

Instructional Engaged Time - Learner engaged times during observations was quite 
high. Most learners appeared to want to learn, seemed to enjoy moving through the 
instructional units, and spent 80%-87% of time in the classroom actually participating in 
learning activities. This compares favorably with engaged times of 40%-50% reported for 
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observations of high school classrooms (Mikulecky). Adult learners came ready to work 
and managed twice as much effort per hour as adolescents manage in school rooms. 

Instructional Quality - The quality of instruction provided by the materials has been 
discussed earlier in the Input section of this evaluation. It was, for the most part, quite 
adequate. All instructors observed had estabhshed good rapport with learners and took an 
active role in monitoring learner progress, encouraging learners, and providing explanations 
when necessary. 

Solid judgements of the quality of insU"uctor explanations of concepts are not 
possible given the fact that explanations and comments to learners were, for the most part, 
privately conducted one-on-one and could be overheard in less than a half dozen instances. 
In these instances, however, a good deal of variation existed in instructor ability to explain 
the thought processes for the job-related basic skills applications procedures being taught. 
One instructor was able to explain several approaches to mathematics in a manner which 
elucidated \ht thought processes involved. The other instructors observed fell back to 
simply repeating procedures from instructional materials, stating step-by-step processes for 
memorization. LitUe or no training was provided to either curriculum developers or 
instructors in how to model such thought processes. This is in contrast with current pre- 
service and inservice practices for workplace literacy instructors in both tlie military and 
private sectors that result in highly effective delivery of instruction tlirough training that 
refocuses instructional delivery practices from the teaching of memorized procedures to the 
teaching of comprehension via modeling thought processes (metacognition). 

Participants were given certificates at appreciation ceremonies held at the plants. 
Minimal attendance was required to receive a certificate, but was reported by project staff to 
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be of high value to participants for recognition of their efforts. In addition, several sites 
gave gifts to the highest achievers. 

For a discussion of project process, please see "Summary of Results" under 
Discussion section of the evaluation. Following receipt of final data, PPLC assessed 
program outcomes (or "product") to determine the degree of project effectiveness. 
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Project Product : 

To what extent are there indicators of project effectiveness ? 

The CLP.P, model enables gathering of evaluation data from more than one source 
to promote triangulation of results in an attempt to arrive at vahd conclusions concerning 
project effectiveness, PPLC evaluated the Greenville Training Opportunities Partnership 
Workplace Literacy Project from three different perspectives of the users: 

• participant pretest/posttest scores and statements concerning 
achievement of personal learning goals and value of the course(s); 

• anecdotal reports from instructors, re^ :rding participant 
applications of course content to work-related and everyday tasks 
outside of class; 

• supervisor pre- and post-course ratings of participants with locally identified 
performance rating indicator snapshots; and, 

• close-out focus groups conducted with project staff and with managers from 
partnering company plants. 

Meeting Participants' Goals - The first aspect of project product effectiveness was the 
collection on pre- and post-program surveys and from on-site interviews of data concerning 
the degree to which participants in the various programs were able to achieve their personal 
^ earning goals. During interviews, most learners expressed satisfaction with the content of 
courses. A frequenUy mentioned asset was the building of confidence that enabled 
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participants to use the skills they were learning in order to pass certification exams or GED 
tests. Participants at two locations mentioned their desire for more structured group work 
and less independent work. All commented on the helpfulness of the insti'uctors. In asking 
learners to rate the program, the evaluator heard that the contents, instructors and schedules 
all earned "A" or "B" grades. Their reasons included liking the classes because they got 
their individual questions answered, well-prepared instructors who seemed to really 
understand participants' needs, convenience of meeting times and locations, and the 
relevance of materials to their personal goals. 

When asked on post-program participant survey forms if they "would recommend 
participation in the program to a co-worker," 100% of the participants responding reported 
that they would. Only 21 post-program surveys were received by the evaluator; these were 
all from JPS plant sites: 9 from Parker, 2 from Slater, 1 from Monaghan, and 9 marked 
only as JPS. No surveys were received from Homelite. The survey forms administered 
were not the PPLC-created form, but a version that had been modified by the project staff. 
Questions pertaining to participant report of goal achievement had been deleted, so no 
information was available on this issue. During focus groups conducted by the evaluator at 
three sites, participants did reiterate many of the reasons they had given for wanting to take 
the course (see Context section of evaluation) as being accomplished. When asked where 
they were applying on the job the skills they were learning, learners responded: 

- have to add and multiply; can't use a calculator 

- have to figure yardage per pound, size formulas; in my job, math happens 
every 2-3 seconds 

- use math doing inventory every month 

- use fractions and rounding off with part dimensions 
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- use a computer for my job 

When asked where they were applying in everyday life the skills they were learning, 
participants responded: 

- in doubling recipes for my family 

- figured out in my sewing how much material I needed to cover a trunk 

- used it in preparing a Sunday School lesson for church 
-calculated the interest on my money at the bank 

- working on my taxes 

- figured out how much interest I'd have to pay if I bought a new automobile 

- knew how much I saved at K-Mart with 25% off sale 

- am building a 4- wheel-drive vehicle from scratch; have to use measurements on it. 

Of those 21 learners completing data collection fomis that asked them to rate their 
program on polarized scales for the program's interest level, usefulness on the job, 
difficulty level, usefulness outside of work, and whether or not it had been what learners 
expected, the results as shown on Table 1 were obtained. 
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Table I: Post-program participants' evaluation of course from analysis of responses to Item on Participant 
Post-Program Survey, n = 21 





5 


4 


3 


2 


1 




Interesting 


91% 


9% 


0% 


0% 


0% 


Boring 


Useful on the job 


33% 


38% 


29% 


0% 


0% 


Useless on the job 


Too difficult 


9% 


28% 


52% 


5% 


6% 


Too easy 


Useful outside 
of work 


57% 


24% 


19% 


0% 


0% 


Useless outside 
of work 


Exactly what 
I expected 


33% 


48% 


14% 


8% 


0% 


Not at all what 
I expected 



From Table 1, one can conclude that 100% of the participants found the programs to 
be extremely to quite interesting. Seventy-one percent found them quite useful for their 
jobs. Fifty-two percent of learners rated the programs as being at the appropriate level of 
difficulty, i.e., neither too difficult nor too easy, while only 15 % found tlie programs to be 
completely out of their range of abilities, rating with a "5" or a "1." Eighty-one percent of 
learners also found program contents to be extremely to moderately useful outside of work. 
It is important to note that 81 % of learners reported that the program was exactly what diey 
expected it would be, and another 29% felt that it was somewhat like what tliey expected it 
would be. This reflects appropriate advertising of program content and goals and is worth 
noting for future program promotions, as well. Please NOTE: Due to the small sample that 
was reported to the e valuator (21 surveys out of 535 participants), these data do not 
represent conclusive findings for the project as a whole. 

Paired sets of participants' pretest-posttest scores were requested from all sites. 
Gains of approximately 50% were reported from the limited numbers of paired sets of data 
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collected for the program math pre- and posttests. No TABE postlest scores were collected, 
since the primary purpose for administering this instrument was placement in appropriate 
program instructional strand. The absence of math posttest scores for many participants, 
due perhaps to high levels of dropping out, make it impossible to determine whether gains 
in math were actually achieved by more than a few program participants. The absence of 
any pre- and posttest reading scores cause gains in that area of instruction to be 
indetemiinable. Differences in individuals' attendance time made comparisons across sites 
also impossible to obtain. 



Table 2; Pretest/ posttest scores reported for program si les. 



Program Site 


TABE 


TABE 


Pretest 


Posttest 


% of Gain 


(No. of 


Reading 


Math 


Math 


Math 


for 


Participants; 


No. of 


No. of 


No. of 


No. of 


Paired 


Average No. of 


Scores/ 


Scores/ 


Scores/ 


Scores/ 


Pre-/Post 


Hrs. Attendance) 


Average 


Average 


Average 


Average 


Math Scores 




Score 


Score 


Score 


Score 


(No.) 


Homelite 


n=16 


n=:16 


n=15 


n=5 


48.75% 


(n=19; 24.47 hrs.) 


20.8 


18.3 


56.8 


92.8 




JPS Slater 


n=21 


n=:21 


n=24 


None 


0% 


(n=24; 12.66 hrs.) 


20.2 


14.8 


indecipherable 


reported 




JPS Parker 


n=53 


n=:53 


n=58 


n=:13 


49.23% 


(n= 58; 14.9 hrs.) 


20.5 


14.9 


44.1 


91.4 




JPS Dunean 


n=29 


n=:29 


n=23 


None 


0% 


(n=: 33; 4.39 hrs.) 


22.0 


14.6 


57.9 


reported 




JPS Monaghan 


n=15 


None 


n=3 


n=12 


52.50% 


(n=14; 41.8 hrs.) 


10.9 


reported 


61.3 


93.5 





Instructor Anecdotal Records - To determine how and if learners were transferring 
new concepts and skills to applications on the job or outside of work, PPLC requested site 
instructors to report any instances of participants referencing situations in which they were 
using outside of class what they learned. An anecdotal recordkeeping form was supplied 
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for this purpose, but was not used by instructors. Information was gathered from only 2 
instmctors. Learner comments included references to: L), improved math ability on 
everyday life tasks, such as figuring percents, measurements, and discounts; 2.), preparing 
to pass entrance tests for college and technical courses; 3.), ability to assist children with 
homework; and, 5.)» increased confidence and decisions to continue with more education. 

Supervisor Ratings of Participants - Pre- and post-program participant ratings by 
supervisors were collected on performance indicator snapshot forms at all five sites. The 
specific behaviors attributed by supervisors to highly competent , average, and less-than 
competent employees in each category were: 

L Dependability 



4 2 



5 

Perfect attendance 


3 

On time and ready for work 


1 

Constantly late 


Always meets or exceeds 
quality standards 


Regular attenaance 


Frequently absent 


Volunteers for overtime 


Stays on job 


Never wants to work 
overtime 


Accomplishes more than what 
he says he will 


Will work overtime when 
asked 


Promises but does not deliver 




Quality of work within 
specifications 


Does not meet 
production/quality standards 

4 




Accomplishes what he says he 
will 






Does what is required 
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11. Team Work 



4 2 



5 

Promotes team spirit in 
department to achieve goals 


3 

Offers to assist others when 
asked 


1 

Refuses to work with others 
in department 


Makes constructive suggestions 


Enjoys and gets along with 
others 


Complains constantly 


Accepts challenges 


Demonstrates concern for 
others' feelings 


Does not enjoy work or 
working with others 


Constantly seeking to improve 
processes 


Works independently; 
participates only as requested 


Wastes time 


Voluntarily gets involved with 
others' problems 


Tries to participate in 
departmental efforts 


Blames others for problems 


Takes charge as a leader 


Reports things that are amiss 
to others 






Has few comments; just does 
his job 






Goes along with others' 
suggestions 
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IIL Self Initiative 





2 


5 

Does not wait to be told what to 
do 


Good attitude; a steady 
employee 


Usually fails to take charge of 
assigned task(s) 


Initiates new ideas or ways to 
improve processes 


Self-starter unless it is a new 
or different job task 


Always waits to be told what 
to do next 


Assists without being asked 


Learns all aspects of current 
job duties 


Always exerts minimal effort 


Requires little or no supervision 


Makes adjustments to new 
ideas and changes with 
minimal resistance 


Never tries to advance 


Requests experience beyond 
scope of present job duties 


Content, comfortable with 
current job duties 


Poor attitude 




Follows instructions 


Shies away from 
opportunities to learn new 
jobs 






Exhibits or expresses some 
distrust of management 
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IV. Job Knowledge 





2 


Assesses progression of work 
in regards to quality and 
quantity stan^ds 


3 1 

Relates problems to 
supervisor 


1 

Unable to asses progression 
of work in regards to quality 
and quantity standards 


Assists ottier workers 


Performs routine job tasks 
well; makes minimal errors 


Always needing h'ilp 


Demonstrates exceptional math 
and reading skills on the job 


Demonstrates good math and 
reading skills on the job 


Runs defective parts 
knowingly 


Makes suggestions for 
improving processes or solving 
problems 


Expresses or exhibits desire to 
progress to new areas of 
responsibility 


Often reacts instead of acting 


Exceeds requirements of job 
performance 


Is knowledgeable about job 


Makes frequent errors while 
using job reading and math 
skills 


Can explain job duties and tasks 
to others 


Is easily trained to perform 
new tasks 


Fails to upgrade technical 
skills 


Works with supervisor to solve 
problems 


Adapts to changes in 
procedures 


Does not understand what to 
do 




Follows quality procedures 


Makes frequent errors while 
performing job tasks 






Requires additional training 
on new or changed 
procedures or tasks 
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V. Adaptability 





2 


5 

Performs unusual tasks 
willingly 


3 

Cooperates as needed; 
performs unusual or new 
tasks with supervision 


1 

Complains when asked to do 
something out of the ordinary 


Demonstrates acceptance of 
change and works hard to see 
that it succeeds 


Helps others without 
complaining 


Performs unacceptable work 
on any unusual or new tasks 


Performs less desu^able tasks 
willingly 


Demonstrates reluctance, 
skepticism about change; 
shows preference to continue 
known tasks 


Exhibits or expresses 
reluctance to take on any 
extra work 


Attempts new things 
enthusiastically 


Exhibits willingness to try 
new things 


Refuses to help others 


Volunteers to help others - 
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VI. Work I'ride 



4 2 



5 

Performs all aspects of work 
conscientiously 


3 

Does not volunteer for 
additional duties, but perfonns 
them when asked 


1 

Abuses company-provided 
equipment or tools 


Produces above quality 
standards 


Produces to quality standards 


Performs below quality 
standards 


Volunteers to works over when 
behind 


Works overtime when asked, 
but does not volunteer 


Refuses to perform additional 
duties 


Demonstrates high self-esteem 




Demonstrates low initiative 


VII. Making Decisions 

4 2 


5 

Judges work as good or bad 
without assistance 


3 

Occasionally makes decisions 
•n own 


i 

Waits until told before 
beginning anything 


Usually makes decisions on 
own 


Exhibits mature behavior 


Shows low concern about 
production or quality 


Uses own judgement about 
when to start next job 


Starts jobs as they are 
assigned 


Exliibits immatuiie behavior 




Asks for approval before 
starting next job 
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VIIL Leadership 



4 


2 


5 

Expresses a desire to excel 


3 

Ixarns to work with others 
and how to instruct if 
necessary 


i 

Gives poor instructions 


Thinks for himself and takes 
action to perform tasks or solve 
problems 


Gathers information from 
others to be able to begin a 
task 


Waits to be told to begin a 
task 


Trains quickly 


Demonstrates willingness to 
learn and asks questions 


Expresses no desire for 
improvement 


Takes initiative to ask questions 
to upgrade job knowledge 


Follows set procedures 
without error 


Work requires additional 
inspection 


Voluntarily helps others 




Blames others; demonstrates 
unwillingness to accept blame 
for own mistakes 


Requests additional training 




Constantly behind in work 


Sees ahead what's happening 
next 




Refuses help from others, 
even when needed 


Takes charge of assigned tasks 
readily 






Is seen to be trusted by 
coworkers 






Exhibits confidence in being 
able to perform job tasks well 
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IX. Taking Responsibility 
4 2 


5 

Demonstrates willingness to 
accept responsibility 


3 

Accepts responsibility when 
asked to 


i 

Exhibits or expresses 
unwillingness to accept 
responsibility 


Volunteers for or requests 
opportunity to perform new or 
additional tasks 


Corrects processes as 
instructed 


Blames others for mistakes 


Looks beyond problems for 
causes 




Waits to be told or shown 
what to do 


Looks ahead to suggest 
improvements 






X, Resolving Conflict 

4 2 


5 

Willing to listen to others' 
views 


3 

Makes suggestions for 
resolving conflict when asked 


1 

Keeps trouble stured up; 
keeps team disrupted 


Is seen as having respect of 
coworkers 


Makes suggestions about the 
causes of a conflict 


Exhibits or expresses 
constant unhappiness with 
his job 


Other workers ask him to 
arbitrate in conflicts 




Demonstrates inability to stay 
focussed on job details; 
becomes emotional 






Demonstrates unwillingness 
to listen to views of others 



Results and impact by site on supervisor-identified attributes are given below site. 
Differences were analyzed by compiling individual ratings for each attribute for each site. In 
general, across all five sites supervisors reported that after participating in the program, 
employees exhibited more of the behaviors attributed to highly competent workers and 
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average workers thaii they had before taking classes. This was evidenced by ratings that 
changed from 2 to 3, 3 to 4, or 4 to 5. 



JPS Monaghan Performance indicators Results (a = 126) 


Categories 


5 

pre/post 


4 

pre/post 


3 

pre/post 


2 

pre/post 


1 

pre/post 


Dependability 


2% / 9% 


57% / 55% 


40% / 35% 


1%/1% 


0% / 0% 


Teamwork 


l%/6% 


11%/ 33% 


62% / 42% 


26%/ 19% 


0% / 0% 


Self Initiative 


0%/ 1% 


7%/ 18% 


40%/ 42% 


52% / 38% 


1%/ 1% 


Job Knowledge 


0% / 2% 


18%/ 36% 


64% / 52% 


18% / 10% 


0%/0% 


Adaptability 


0%/l% 


2% / 9% 


47% / 52% 


49% / 37% 


2%/ 1% 


Work Pride 


0% / 2% 


20% / 39% 


60%/ 48% 


19% / 10% 


1% / 1% 


^^ake Decisions 


0%/\% 


2% 11% 


41%/ 52% 


56% 40% 


l%/0% 


Laidcrship 


0% / 0% 


2% 1 5% 


29% / 34% 


69%/ 61% 


0% / 0% 


Take Responsibility 


0% / 0% 


0% / 32% 


54% / 56% 


45%/ 12% 


l%/0% 


Resolve Conflict 


0% / 2% 


l%/40% 


54%/ 48% 


40%/ 10% 


5%/0% 
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JPS Dunean Performance I 


ndicators Results (n = 72) 


Categories 


5 

pre/post 


4 

pre/post 


3 

pre/post 


2 

pre/post 


1 

pre/post 


Dependability 


42% / 37% 


39<,o / 40% 


19%/ 7% 


0% / 0% 


0% / 0% 


Teamwork 


8%/ 16% 


47% / 47% 


39% / 20% 


6% 71% 


0% / 0% 


Self Initiative 


8% / 10% 


23%/ 31% 


39% / 66% 


3% 74% 


0% / 0% 


Job Knowledge 


19%/ 23% 


50% / 43% 


31%/ 18% 


0% / 0% 


0%/0% 


Adaptability 


3% / 7% 


29% / 39% 


61%/ 37% 


7%/ 1% 


0% / 0% 


Work Pride 

1 


12%/ 18% 


54% / 47% 


32% / 19% 


l%/0% 


l%/0% 


Make Decisions 


0%/4% 


23% / 26% 


67% 751% 


10% 3% 


0% / 0% 


Le{Klership 


0% / 1% 


23% / 28% 


51%/ 46% 


25%/ 9% 


l%/0% 


Take Responsibility 


3% / 7% 


32% / 34% 


45%/ 38% 


19% / 5% 


l%/0% 


Resolve Conflict 


4%/ 4% 


20% / 29% 


57%/ 48% 


18%/ 3% 


1 l%/0% 

1 



(16% of Evaluations were not completed) 
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JPS Parker Performance Indicators Results (a = 74) 


Categories 


5 

pre/post 


4 

pre/post 


pre/post 


2 

^re/post 


1 

pre/post 


Dependability 


30%/ 40% 


49%/ 49% 


17% / 1% 


4% / 0% 


0% / 0% 


Teamwork 


30%/ 47% 


35%/ 42% 


27%/ 1% 


8% / 0% 


0% / 0% 


Self Initiative 


15%/ 35% 


39%/ 51% 


39% / 4% 


7% / 0% 


0% /0% 


Job Knowledge 


21%/ 37% 


48% / 53% 


28% / 0% 


3% / 0% 


0%/0% 


Adapiabilily 


21%/36% 


42%/ 51% 


32% / 3% 


5% / 0% 


0% / 0% 


Work Pride 


26%/ 40% 


41%/ 50% 


29% / 0% 


4% / 0% 


0% / 0% 


Make Decisions 


16% / 23% 


29%/ 64% 


40% / 3% 


15% 0% 


0% / 0% 


Leadership 


25%/ 29% 


21%/ 56% 


43% / 5% 


11% /0% 


0% / 0% 


Take Responsibility 


21%/ 26% 


38%/ 62% 


33% / 2% 


8% / 0% 


0% / 0% 


Resolve Conflict 


10%/ 23% 


33%/r^5% 


41%/ 1% 


13% /0% 


3%/0% 



(10% of Evaluations were not completed) 
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JPS Slater Performance Indicators Results (a = 79) 


Categories 


5 

pre/post 


4 

pre/post 


3 

pre7post 


2 

pre7post 


1 

pre7post 


Dependability 


19% 7 32% 


42% 7 47% 


34% 7 15% 


4% / 0% 


1%70% 


Teamwork 


10% / 19% 


27% 7 46% 


54% 7 27% 


9% / 2% 


0%70% 


Self Initiative 


3% / 16% 


26% 7 34% 


54% 7 39% 


16%/ 5% 


1%70% 


Job Knowledge 


11%/ 17% 


30% 7 57% 


51% 720% 


8% / 0% 


0%70% 


AdapUibilily 


4% 714% 


28% 7 39% 


58% 741% 


9%/0% 


1%70% 


Work Pride 


10% 722% 


29% 7 40% 


55% 7 32% 


6% 70% 


0%70% 


Make Decisions 


5% 7 11% 


23% 7 27% 


54% 7 56% 


17% 0% 


1%70% 


Leadership 


4% 7 10% 


16% 7 25% 


53% 7 55% 


26% 7 4% 


1%70% 


Take Responsibility 


5% 7 12% 


23% 7 32% 


58% 7 47% 


13% 7 3^ 


1%70% 


Resolve Conflict 


1%7 8% 


21% 7 30% 


62% 7 55% 


15% 71% 


l%70% 

1 



(6% of evaluations not completed due to attrition) 
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Homelite Performance Indicators Results (n = 50) 


Categories 


5 

pre/post 


4 

pre/post 


3 

pre/post 


2 

pre/post 


1 

pre/post 


DcpeiKlability 


8% / 20% 


54%/ 40% 


34%/ 18% 


4%/0% 


0%/0% 


Teamwork 


10%/ 15% 


40%/ 49% 


44% / 24% 


6% / 0% 


0% / 0% 


Self Initiative 


6% / 12% 


16%/ 30% 


56%/ 34% 


20% / 2% 


2% / 0% 


Job Knowledge 


4% / 8% 


45%/ 46% 


37% / 24% 


10% /0% 


4% / 0% 


Adaptability 


4% /14% 


36% / 39% 


50% / 23% 


10% / 2% 


0%/0% 


Work Pride 


10%/ 18% 


52% / 50% 


32% / 10% 


6% / 0% 


0% / 0% 


Make Decisions 


0%/4% 


19%/ 32% 


55% / 40% 


24% 2% 


2%/0% 


Leadership 


2%/ 2% 


20% / 30% 


48% / 40% 


28% / 6% 


2% / 0% 


Take Responsibility 


2% / 6% 


34% / 34% 


48% / 38% 


16% /0% 


0%/0% 


Resolve Conflict 


0% / 0% 


18%/ 28% 


60% / 50% 


20% / 0% 


2%/0% 



(22% of evaluations were not completed) 



Outcomes from Close-Qut Focus Groups with Project Staff and with Partners : During the 
focus group conducted with the project staff, the following issues were raised and 
discussed. 

• Continued provision of services to the partnering company sites in the 
future- Homelite has new plant manager; thought they would want to 
continue program. JPS Dunean probably will not continue due to poor 
attendance. Monaghan probably will continue, but without recordkeeping 
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strictly on vpluntary employee time. Parker and Slater JPS sites unknown. 

• Problems mth attendance' Instructors expressed concern over the 
"tremendous" drop-out rate and attributed it partially to tlie logistics of 
rotating shift times and participants not always scheduled to come directly 
before or after a work shift. One instructor thought after shifts was the best 
time to maximize attendance. Another noted that payday interfered with 
class time remaining a priority. The Dunean instructor reported tliat none of 
her students could attend class more than once per week and that this made 
it difficult to keep participants motivated. Suggestions for improving this 
problem included paying participants each time they completed a segment of 
instruction, such as a workbook, or using more computer-based instruction, 
or operating all classes on clock time and for shorter structured cycles, like 
the SPC Prep Mini-math courses, rather than open-entry/open-exit as they 
currently run. 

• Using counselors in the program to aid comfort levels and retention of 
participants- The staff described several examples of specific 
instances when counseling had uncovered potential problems and solved 
them, such as identifying a deaf pm*ticipant and obtaining a paid translator 
to assist; retaining several participants by explaining the difference of indivi- 
dualized instmction versus whole group instruction; and, calling instructors 
to ensure that a shy participant received extra attention when needed. The 
counselors and director, however, felt that overall the counseUng component 
had not been effective. It may have resulted in the retention of approximately 
25% of potential program drop-outs, but was designed more for those who 
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were chronic absentees rather than the attenders. The counselors each 
visited classes on a regular basis, but found that those participants who 
needed their services most,were the ones who were not in class. Additional 
problems were perceived to be the inability to discuss participant problems 
with mstructors during class visits because of the quiet nature of independent 
study; and the discomfort of the participants with using counselors. 
Staff reported that when participants were not coming, it was for reasons 
that were exten.al to the program, usually stemming from life and family 
constraints that precluded continuing to pursue learning. 

The focus group conducted with the managers representing each of the partnering 
plants addressed issues of concern to them, namely: 

• Lack of employee participation in the program voluntarily- Managers 
commented that share time or monetary incentives might have increased 
the number of participants. Being a "no frills", low-budget industry, 
they feel unable to pay for full release time for employees to participate in 
the program. Several Micd the difference in attendance for the SPC Prep 
course and for the original sti ands of the program. They offered examples 
of employees who had given verbal praise for the program, but did not 
have any quantifiable measures for determining cost benefits at the time of 
the focus group. Without some form of job impact data, the managers 

felt that they could not request release time from higher levels of management 

• Selection of site-compatible instructors- Several managers commented on 
the lack of worksite political sophistication exhibited by several of the 
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instructors while on site. They felt that having more input into the criteria 
for selecting instructors or into the required training for them or in partici- 
pating in the hiring/ interview process would have resulted in a better "fit" 
between plant personalities and needs and the program instructors. 

Discussion 

Limitations of this study - Tv/o factors limited the ability of this study to draw definitive 
conclusions from the evaluation. The first was the inconsistency of data able to be collected 
across sites. The budgeted time and financial resources allotted to project evaluation, i.e., 
10 staff days, resulted in insufficient time available for thorough and complete investigation 
of aspects concerning each progiam delivery site. 

The second limiting factor was the difficulty experienced by the evaluator in 
collecting and obtaining some of the requested data from the program providers in the 
fomiats required for inclusion in the evaluation. Unfortunately, the absence of some key 
program measures that were requested throughout the project severely limits the evaluator's 
ability to draw conclusions about the overall effectiveness of this demonstration. 

Summary of Results - The following statements provide summary and discussion of key 
findings from the evaluation of project context, input, process, and product. 

Context - The extent to which the goals and philosophy of the project were shared by key 
project personnel and learners was found to be as follows: 
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Areas of consens us: There was a good deal of consensus about program goals 
among the project director and the program curriculum developers. All highlighted the 
importance of the instruction as a means for mastery of basic skills for both everyday life 
and applications to job tasks and requirements. Participants also commented on their desire 
to improve these skills and on the programs' relevance to accomplishing their personal 
goals. 

Areas of divergence : The main areas of divergence were evidenced during 
interviews in the responses from the instructors and tlie plant managers, namely their 
reluctance to commit to goals for totally job-relevant programs. They all commented on 
their desire to make the programs more oriented toward worker self-actualization , rather 
than adhering to the program's published purpose of providing the job-specific workplace 
programs agreed to by the college and the partnering companies in order to obtain the grant 
monies. There appeared to a lack of understanding among the instructors and among the 
managers about the nature of the project and of a functional context design for workplace 
literacy instruction. The existence of such mixed philosophies between the curriculum 
developers and instructional deliverers and business partners was in direct conflict with the 
developer's goals for the project. 

These observations should not be taken to mean that instructors were not doing their 
jobs or that managers did not involve themselves with program activities. Participants 
expressed indications that they perceived themselves to be learning skills they could apply in 
the workplace and everyday activities and were having their needs met. Most learners were 
satisfied with theu* experiences, sometimes because of instructor personal attention. 
Managers also reported checking with individual participants to monitor program activities. 
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It may be, however, that the orientation activities for the project staff and 
management partners vv^ere less tlian adequate. The informal and minimal pre-service 
training provided for instructors was insufficient to guarantee their internalizing program 
goals. With such brief and erratic training, it is likely that instructors will maintain whatever 
learner goals they have used previously. With development of instructor training materials 
carefully structured to include guidance, modeling, and post-training assessment of delivery 
skills, this problem might have been alleviated. 



Additionally, the absence of mention of work-related program goals by most of the 
managers caused identification of cost benefit indicators for benchmarking and measuring 
program impact on the organization very difficult, if not impossible. This meant that the 
program outcomes were only perceived and evaluated as education for individuals, ratlier 
than as value-added training benefitting the organization as well as for the individual. As 
such, there is an inherent danger that the program is, at best a charitable endeavor and at 
worst, a duplication of existing community education services for which the companies 
akeady pay tax dollars. In addition, if piani managers do not perceive the program to be of 
value to the organizational goals at their site, workers may also doubt its value. Conducting 
a program on employee volunteer time adds to this program devalued image, as well. The 
seeming lack of understanding among plant managers of the general piupose of the 
workplace literacy demonstration and of their need to identify arid document impact on their 
organizations might have been avoided by a more comprehensive presentation of grant 
purpose and requirements to potential business partners prior to submitting the grant 
proposal. 



Input - ITie availability to die project of resources during development and implementation 
and to what extent they were effectively used was found to be as follows: 
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Strengths aiid Weaknesses - Resources for program development appeared adequate 
financially for instructional delivery, and realistic materials development time lines impacted 
little on the stress level of the developers. The curriculum materials developed for the 
program, however, contained few job task examples of skill applications to enable learners 
to practice skills in ways they would use for the workplace. The content of several units of 
instruction in the customized course materials included a number of previously published 
excerpts from other commercially available sources. Using others' words and ideas 
anonymously, without first obtaining written permission from the copyright holders and 
authors, is a major flaw in the curriculum design and development of these particular 
programs. Inclusion in subsequent publishing or dissemination of project curricula could 
also lead to legal ramifications for the project administrators. 

Additionally, the absence of pre- and posttests in the customized courses precluded 
the collection of conclusive evidence on curriculum materials effectiveness. Writing 
customized pre- and post-tests to include in the reading and matli courses developed for JPS 
and Homelite w. uld have enabled measurement of participant mastery of what had been 
taught. Several instructors expressed slight dissatisfaction and confusion with some of the 
customized functionally contextual curriculum materials. Had the developers provided a 
more in-depth instructor training and instructor scripts and guidelines to accompany the use 
of the customized curriculum and integrated commercial materials, this problem might have 
been overcome. 

Counseling services for participants were provided to enhance retention. Project 
staff indicated that the attendance of counselors to class sessions proved to be an impractical 
use of this resource because of the real need to be counseling absentees rather than attendees 
and because of the difficulty in building rapport with participants in classroom settings. 
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Either using the counselors as off-site visitors to follow-up on absences or creating an actual 
worksite contacv location out of which counselors could operate on a regular basis might 
have improved the quality and effectiveness of interactions with program participants. 

Instructor, counselor, director and curriculum developer qualifications and previous 
experience were rich and highly professional; they provided a definite enhancement to the 
program overall. Criteria might be derived from a composite profile of tlie qualifications 
and background of these key personnel for use as hiring guidelines for project or program 
institutionalization or replication. 

Instructor training sessions proved to be inadequate; they did not provide the total 
support system that the program needed for full acceptance by the instructors, congruence of 
purpose and mastery of instructional techniques. The brief, informal deliveries of program 
overviews addressing the functional context workplace literacy curricula did not 
satisfactorily meet the needs of project personnel for preparing to become workplace 
instructors or for ongoing support necessary for part-time, geographically isolated staff. 

Process - The extent to which program development and observed instmction were 
congruent witii program goals and research on instructional effectiveness follows: 

Areas of convergence and divergence : Learner engaged time was quite high and 
learners spent 80-87% of time in the classroom actually participating in skill building 
activities. Both instructors and pexticipants appeared motivated to take full benefit of 
instruction time and took pride in the efforts made. 

The quality of instruction was good overall. Each instnictor that was observed 
appeared to be engaged in "reciprocal learning" with the learners and displayed a caring 
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attitude and willingness to assist learners achieve their goals. Evidence varied from 
instructor to instructor in ability to demonstrate the thinki)»2 processes necessary for 
applications of workplace basic skills to be taught effectively. 

The project well exceeded its goal for providing assessment for 600 workers and 
services to approximately 450 participants with records of assessments of 1 134 workers and 
total enrollments of 535 workers. This is a strong indication that overall, recruitment and 
everyday relationships with partnering organizations were operating well. The programs 
that repoited dwindling attendance were those designed to provide open-entry/ open-exit 
self-selected individual study on a voluntary basis. This may have been due to the generic 
content of those courses not directly focused on transfer of learning to improvement of 
performance of job applications of basic skills. The participants may not have been aware 
of the connection of workplace literacy with training, i.e., the impact on organizational 
improvement as well as individual improvement, in contrast witli education which impacts 
only on the individual. In addition, it may reflect on the seeming inappropriateness of 
design of unstructured, lengthy commitments to educational programs for use by working 
I dults. The workplace programs that are most effective in retaining participants are those of 
limited duration, e.g., 7-10 week instructional cycles that are manageable for short-tenn 
integration with participants' daily schedules, and programs that are direcdy linked to 
supporting organizational training needs. 



Product - The impact of the program was assessed with a combination of indicators, 
including comments from learners and instructors, and supervisor ratings. A summary of 
the results follows. 



Prepared by Performance Plus Learning Consultants, Inc. 



ERIC 



11^ BEST 



GreavjiUe, SC Traintig Opportunities Portnersl'iip 

DOE Workplace Literacy Grant Program- Final Evaluation Report 



Business and industry organizations noiTnally evaluate training on four levels. Because 
workplace literacy programs are directly related to assisting workers attain career goals by 
meeting job requirements and improving performance on job tasks, it is appropriate to 
measure program outcomes using tiiis yardstick: 

Level I- does the proposed program match with an identified organizational need? In tills 
case, the project programs were desired by each of the partnering organizations to enable 
their members to function better through improved workplace applications basic skills. The 
grant application shows diat specific job tasks and special needs of each cooperating 
organization were to be identified and targeted. The job tasks and requirements were 
carefully selected and analyzed tiirough DACUM job analyses, the results of which were to 
be utilized for customized workplace literacy curriculum development. 

Level II- do the participants selected for training master the content of the training program? 
Absence of sufficient numbers of paired sets of data from participant scores on pre- 
/posttests resulted in the need to turn to ancillary sources for evidence of master)' of content. 
Limited numbers of instructor anecdotal reports and post-program statements by participants 
compared to pre-program goal statements, provide inconclusive evidence that participants 
mastered the content of the programs for which this data was collected. 

Level UI- do those participants who master training demonstrate improved job performance 
in areas identified as critical to show positive transfer of learning? Comparing pre- and 
post-program local performance indicator snapshot ratings of participants by tiieir 
supervisors indicated that in every category of the identified performance attributes 
supervisors noticed positive changes in employee performance. Supervisors who saw 
improvements were able to identify specific observable, measurable behaviors tliat 
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demonstrated positive transfer of instruction to job tasks. Because the actual content of 
course materials does not directly address the observed job applications, and because of the 
lack of adequate numbers of pre- and posttest scores collected, it is impossible to determine 
whether participant improvements in performance are the result of mastered learning or of 
improved self-esteem. 

Level rV - does impact on performance lead to demonstrable cost benefits, /.e., money 
saved or generated, by the positive changes in employee behavior? In this c^se, the 

organizations indicating positive program impact did not report any cross-referencing of 
individual productivity or behavioral indicators with perfomiance appraisals, the supervisor 

ratings and instructional objectives of the programs. No data exists, therefore, for 
determining the possible cost-benefits derived from employee participation in the programs. 

The absence of actions under consideration, or already taken, by some of the partnering 

business organizations to continue programs beyond the grant period, along with lack of 
evidence that managers understood the goals and purpose of the program, does not indicate 

company perception of or satisfaction with services as beneficial for their organizations. 
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Conclusions and Reconimendations 



Based on the results of this evaluation, the following conclusions and 
recommendations concerning stated grant goals are offered. 



There is strong evidence showing: 



that the project programs implemented targeted and delivered services 



to a minimum of 150 workers at each industry location and using 



initial assessment scores to identify eligible program participants. 



There is only a moderate amount of evidence indicating: 

• evidence of successful program implementation through tiie use of 
appropriate processes for participant recruitment, class scheduling, 
development of individual education plans, curriculum delivery, pre- and 
post-assessment, and instructor training and support, that are academically 
and organizationally sound and that match with program goals. 

• evidence of the offering of additional services to each project participant 
through the use of an individualized program of literacy instruction, 
supported by personal counseling, tutorial services, child care, and/or 
transportation assistance, and subsequent course completion by 60% of 
participants or more. 

• evidence of a reduction in the incidence of workplace literacy in Greenville 
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County through the establishment of workplace literacy partnership 
programs at three area companies and assessment of at least 200 employees 
of each company. 

Recommendations : 

1 . Strengthen curriculum delivery through structured instructor pre-service 
and inservice training that addresses the differences between customized 
functionally contextual materials and academic basic skills materials. 
Provide modeling of instructional techniques and an overview of 
required recordkeeping and data collection activities. Create pre- and 
post-assessment instruments for each customized course. Meet with 
instructors to develop local indicators for ranking participant progress 
with GED and commercial materials. Have instructors create and use a 
system of portfolio assessment with weighted captions so that samples of 
achievement will be available to measure progress for participants who 
are persisters or leavers of the program. 

2. Improve the delivery of counseling services to focus efforts on working 
absentees rather than with class attendees. Establish worksite locations 
for delivery of ancillary, on-going counseling services to targeted 
program participants. Divert unused funds from child care and 
transportation support services to other areas of program development 
and implementation. Lxx)k for altemate methods to enhance retention of 
participants, such as 1., structuring classtime into smaller cyclical units 

* to suggest closure on instructional topics in shorter periods of time; aiid, 
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2., promoting the program at worksites as value-added (HRD) training 
that is directly related to and impacts on the company(s) as well as the 
individual workers, rather than as . iucation that benefits only the 
individual. 

3 . Comparison of supervisor pre- and post-program performance indicator 
snapshot ratings of participants indicate positive change in observed 
employee behaviors; pre- and posttests for the content of the customized 
insuoictional materials need to be developed in order to demonstrate that 
the content of instruction was mastered and is related to the cause of 
diese improvements due to positive transfer of learning to job 
performance. 

There is little or no evidence showing: 

evidence of the development of a training methodology for adult learners that 
meets specific workplace needs, with individualized materials for each 
i -^Ty that utilize that industry's available job-related materials. 

evidence of a smooth instructional flow of activities witliin the developed 
curricula, reflecting a sound developmental approach to mastering those 
literacy skills necessary for competent performance of identified job tasks. 

evidence that measu^'^ment of the impact of training on project participants 
was conducted via internal evaluation processes that tracked 1, literacy skill 
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achievement by means of pre- and post-test scores; 2, job productivity; and 
3, personal growth and development. 

evidence of a county-wide awareness campaign about adult literacy 
conducted through various local media and of a final project rejJort on 
methodology and results available for national dissemination. 

Recommendations : 

1 . Use the results of the DACUM in conjunction with actual observations of 
job tasks being performed by competent workers. Analyze the 
applications of basic skills and thinking strategies as seen in competent 
performance of organizationally critical job tasks. Model these job 
applications of literacy skills within each unit of instruction, rather tiian 
modeling applications of skills on everyday life or academic examples. 
Demonstrate step-by-step instruction in die metacogiiitive aspects of 
transfer of learning to new applications, focussing on workplace 
examples in both direct instruction and practice exercises. Emphasize the 
how of skills application processes, rather than the what of content or 
format. Customize curriculum design for each company instead of using 
an academic context for the instructional blueprint tiiat appends only a 
minim'TO of workplace-specific examples to lessons. 

2 . Develop original materials based on the critical tasks identified by each 
partnering company in the up-front job analysis. If materials from other 
existing sources need to be integrated as occasional examples, use them 
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sparingly after obtaining pemiission; always identify the source(s) of 
such materials, wherever they appear. 

3 . Pretest scores and counselor statements written on intake forms 
demonstrate project interest in benchmarking 3tart-up condidons. Follow 
through with the diligent collection post-program data in order to provide 
evidence of participant progress toward mastery of instructional 
objectives and development of personal growth. Job productivity is 
difficult to establish without company agreement(s) to identify, collect, 
and make available information on individual productivity indicators. 
The local performance indicator snapshot ratings satisfy the need for 
evidence that indicates transfer of learning to job performance and 
support of organizational goals. The addition of pre- and posttests to the 
customized curricula would more strongly connect improved 
performance to program content. 

4. Prior to contacting local media, be certain that community awareness has 
been built on issues of workplace literacy and its purpose in the context 
of a broad-based definition. This will eliminate some of the negative 
connotations accompanying words like literacy and basic skills. Be sure 
to obtain company and participant permission before inviting media 
coverage of the program. Work with local reporters, the Chamber of 
Commerce, community service organizations, etc. to encourage a 
community-wide "team" approach to bettering economic conditions for 
all citizens through support and interest in programs such as yours. 



Prepared by Performance Plus Learning Consultants, Inc. 

110 



MEmODOLOGY GUIDE 



120 

ERIC 



T. 0. P. 

(TRAINING OPPORTUNITIES PARTNERSHIP) 



CONTENTS 

INTRODUCTICW 1 

♦ T. 0. P. PROJECT GOALS 4 

FLOW CHART 6 

PROJECT METHODOLOGY 8 

NEEDS ANALYSIS OR JOB TASK ANALYSIS 10 

CURRICULUM DEVELOPMENT 16 

EMPLOYEE ORIENTATICW 17 

ASSESSMENT ,,,,,,, 18 

TRAINING 20 

SUPPORT SERVICES 21 

TRAINING COMPLETiai 23 



FUNDED BY: 

National Workplace Literacy Program CFDA No. 84.198A 

U. S. Department of Education 

1991-1992 

(Federal share $177,724 [70%]; College share $87,730 [30%]) 
Copyright 1992 

ERIC 



iNTRODUCTION 



INTRODUCTION 



Ihe world of work is changing. Technological advancement and global 
economic forces are redefining the Anverican workplace. Coitpaters and robotics 
now allow fewer workers to do the work once required of many. Global 
coitpetition demands maximum productivity and efficiency of American coitpanies. 
Increased enployment opportunity in service-based industries has thrust xoore 
workers into managerial participation. 

The common element in this changing workplace environment is the 
requirement of higher le' .j.s of education tham demanded in jobs of the past. 
The new technology must be understood. Information must be analyzed and 
decisions made. Production and control systems are no longer a purely 
administrative domain. 

Into this workplace comes a workforce ill-prepared to answer its demand. 
One of every five American workers reads at or below the eighth grade level. As 
many as 80 percent of job applic§uits are unable to pass basic enployment exams 
in English and math. Between 20 and 40 million adults are believed to be 
illiterate. 

The economic and business costs of this problem are severe. Those vrtio do 
not have basic skills may have difCicxalty in finding and keeping jobs. On the 
job, billions of dollars are lost vrtien workers make costly mistakes or receive 
injuries when instructions or warnings cannot be read or xanderstood. The 
biggest cost of illiteracy, however, may be the one that is hardest to measure: 
the loss of self-esteem felt by a worker who is undereducated and undertrained. 

In tha past, literacy at work was not the paramount issue it is today. The 
dramatic need for basic skills enhancement has been addressed by the National 
Workplace Literacy Program, vrfiich requires a cooperative partnership between 
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education and industry to develop and demonstrate innovative work--based literacy 
programs. 

The Training Opportunities Partnership (T.O.P.) is a workplace literacy 
project funded through the National Workplace Literacy Program for the project 
year 1991. The Department of Education grant was awarded to Greenville 
Technical College in partnership with five manufacturing industries. The 
program goals and methodology are the topic of this report. 
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T.O.P. PROJECT GOALS 



!• To iinprove the productivity of the labor force through inproveroent of the 
basic literacy skills needed in the workplace; and 

2. To enhance vvorkplace literacy by collaborating with industry: 

- to provide the resources necessary for literacy training 
in the workplace; 

- to reach enployees v^o otherwise would not have access to literacy 
programs. 

Several measurable objectives were established to achieve the project goals: 



OBJECTIVE 1 



OBJECTIVE 2 



OBJECTIVE 3 



OBJECTIVE 4 



OBJECTIVE 5 



OBJECTIVE 6 



OBJECTIVE 7 



OBJECTIVE 8 



To reduce the incidence of workplace illiteracy in Greenville 
County by establishing a partnership with three area conpanies that 
will provide for the assessment of at least 200 eitployees of each 
coii?>any during the eighteen-month project period. 

To develop a training methodology for adult learners to meet 
specific workplace needs. Training materials will be 
individualized for each partnership industry and will utilize tliat 
industry's available job-related materials. 

To offer the developed literacy training to at least 150 eirployees 
of each partnership industry during the project period. The 
specific training suggested for each targeted enployee will be 
based on that employee's initial assessment scores. 

To offer additional services to each project participant through 
the use of an individualized program of literacy instruction that 
is supported by personal counseling, individxoal and/or group 
tutorial services, child care, and/or transportation assistance. 
These additional support services are expected to increase the 
average course coitpletion rate from 50 percent to at least 60 
percent or more. 

To measure the impact of training on project participants through 
an evaluation process of tracking skill achievement, job 
productivity, and personal growth and development. 

To evaluate the impact of project training on the partnership 
industries through an assessment by management and/or supervisory 
personnel of individual eirployee productivity. 

To evaluate the project methodology by determining original 
literacy levels of project participants through initial assessment 
scores and corqparing these scores to post-training test results. 

To prepare a final project report on the methodology and results of 
the project that will be available for dissemination to interested 
groups and institutions statewide and nationwide. 
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PROJECT METHODOLOGY 



During the grant proposal phase of the T.O.P. program, local industries 
were identified to serve as partners in the cooperative demonstration project* 
At that time, three partners indicated their commitment to participate. At the 
time of funding approval, one industry foxmd it necessary to withdraw from the 
program. This allowed the expansion of the program to additional sites, 
resulting in the inclvision of five manufacturing industry partners. They were 
as follows: 

JPS Converter and Industrial Corp. - Monaghan Plant 
JPS Converter and Industrial Corp. - Dunean Plant 
JPS Converter and Industrial Corp. - Slater Plant 
JPS Automotive Products Corp. - Parker Plant 
Homelite Division of Textron, Inc. 

Implementation of the program was completed utilizing the steps presented and 

explained here. 
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1. Needs Analysis or Job Task Analysis 

To identify the specific literacy training needs of each conpeiny^ an 
intensive needs analysis was performed on site utilizing the "Bull's Eye" 
Approach to Specialized Program Developtoent. This method involves a DACUM 
(Developing A Curriculxam) process that relies on input from both management and 
labor. Workshops are held on two consecutive days. On the first day^ management 
representatives tackle the task of identifying the specific literacy skills used 
on the job. The product of this listing is a matrix of skill requirements. 
This matrix is presented on the second day to hourly personnel representatives 
who modify or approve what the management team has suggested. A copy of a DACUM 
chart is included here. 
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HOMELITE T.O.P. PROGRAM 
NEEDS ASSESSMENT 



MAJOR 


SKILL APPLICATIONS 


COMPETENCY 






A. READING 


1 2 


3 




Safety Rule General Plant 


Safety and Chemical 




Sheets Rules 


Warnings (House signs) 




4 5 


6 




Chemical Training Employee 


ELF System Termi- 




Sheets Handbook 


nology (alphabet 






and numbers) 




7* 8** 






Plant Terminology Forms 


Charts & Graphs 




(recognize and 






understand) 
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Bulletin Boards Plant Newsletter 


Material Reference Books 




PLANT TERMINOLOGY 






*SPC 


Diamond 




Blue Prints 


Inserts 




Process Sheets 


Boring bars 




Work Orders 


Drills 




Routing Sheets 


Reamers 




Pareto charts 


Countersinks 




AQL 


Feeler gauges 




Standard Deviation 


Basic Tool Names 




Mean 


Rockwell 




Range 


Shems 




Tenths 


AA 




Thousands 


Geometric Tolerancing 




Millionths 


Gibbs, Ways 




Coolant 


Dykem 




Safety data sheets 


Indiron 




DMR 






Productivity Standards 






and Performance 






Utilization 






FORMS 


CHARTS AND GRAPHS 




**T & P Tickets 


^Quality Assurance 




Operation Routing Cards 


charts and graphs 






ridi i^Ajwc I XT L uuuu L 1 V 1 Ly 




Insurance 


Performance 




Special Handling Tickets 


Utilization 




Rejects 






Directions on machines 






MATERIAL REFERENCE BOOKS 












CRT Commands (computer language) 
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HOMELITE T.O.P. PROGRAM 
NEEDS ASSESSMENT 



MAJOR SKILL APPLICATIC^S 

COMPETENCY 



B. WRITING 


1 2* 


3 




Plant Terminology Forms 


Idea Expression 




4 

Writing in Logs 




FORMS 






*Scrap Reports 


Gate passes 




Defective material reports 


Group production tickets 




Time and production reports 


Gauge tickets 




Status reports 


Fabrication tickets 




Work orders 


Disability forms 




Tool requisitions 


Unemployment forms 




Shift comiTTunication memos 


Payroll deduction forms 




Process operating details 


Savings bond 




SPC charts 


W-4 forms 




Pareto charts 


Application forms 




Equipment damage reports 


1-9 forms 




Gauge requisitions 


Routing tickets 

"Let's Communicate" forms 




Quality circle presentation 




Corrective action interview 


Insurance forms 




Stock forms 


Vacation requests 




Inventory tickets 


Job Bid forms 
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HOMELITE T.O,P. PROGRAM 
NEEDS ASSESSMENT 



MAJOR 
COMPETENCY 



SKILL APPLICATICX^S 



COMPUTATICX^ 



2** 

Add, Subtract 
Multiply, and 
Divide 



Fractions and 
decimals 



Counting 



4 

Understand 
Positive (+) and 
Negative (-) 
values 



Scales and 
Measurements 



Percentages 



7 8 

Tolerance - range 24 hour time clock 

variance system 



Reading graphs 
( tangents , radius , 
angles ) 



COUNTING 
mparts 
Figuring hours on job 
Figuring Production (Rejects/Keworks) 

BASIC COMPUTATION 
**Figuring Production (subtraction/multiplication) 
Using gauges to check parts (division) 
Recording Inventory 
Scheduling production 
Checking production (averaging) 
Figuring machine adjustments 

Determining parts required for assemblies (overages or 

shortages) 
Figuring dimensions (Blueprints) 

DECIMALS 

***Reading decimals to make machine settings/adjustments, 
using blueprints, process sheets 

SCALES AND MEASUREMENTS 



****Micrometers 
Caliphers 
Gauges 
Scales 

Volume and weight 



Comparators 
Arithmatical Average 



(surface finish) 
Inches, feet 
Metric and US 



PERCENTAGES 

*****Under standing how percentages are figured 



Figuring payroll 



Figuring production 



ERLC 
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HOMELITE T,0,P. PROGRAM 
NEEDS ASSESSMENT 



MAJOR 
COMPETENCY 




SKILL APPLICATIONS 




D. ORAL 

COMMUNICATION 


1 

Basic Grammar 


2 

Comprehension 
"Listening 
Skills" 


3* 

How to speak up, 
present ideas, be 
assertive 




4 

Ask questions 


5 

Direct terms 
for clarity 


6 

Job interviews 
(promotion/transfer) 




EXAMPLES 

Explain problem to co-worker or supervisor 

Explain physical problem to nurse 

Share shift change inforirtation 

Explain operations/procedures to co-worker 
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HOMELITE T.O.P. PROGRAM 
NEEDS ASSESSMENT 



APRIL 29 AND 30, 1991 



FACILITATOR: Joan Mason, Greenville Technical CollegF- 

Helen Clarkson, Greenville Technical College 

RECORDER: Al Stokes, Greenville Technical College 

PANELISTS: Ed HincJman, Personnel Manager 

Homelite, Division of Textron, Corp, 

Susie Fontenot 

Homelite, Division of Textron, Corp. 

Gene Phillips 

Homelite, Division of Textron, Corp. 

Lee Cox 

Homelite, Division of Textron, Corp. 
John Pruitt 

Homelite, Division of Textron, Corp. 
Merle Arseneau 

Homelite, Division of Textron, Corp. 
Tim Howell 

Homelite, Division of Textron, Corp. 
Nick Zatcrsky 

Homelite, Division of Textron, Corp. 
Jennie Weathers 

Homelite, Division of Textron, Corp. 
Gay Henson 

Homelite, Division of Textron, Corp. 
Lib Hainmett 

Homelite, Division of Textron, Corp. 
Donna Mills 

Honelite, Division of Textron, Corp. 
Jackie Cantrell 

Homelite, Division of Textron, Corp. 
Mable Pendergrass 

Homelite, Division of Textron, Corp. 
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2. Currioilura Development 

The final training needs chart is the primary source of information used by 
the course developer in preparing the functional context curriculiam appropriate 
to each training site. VJhere necessary for greater clarity in identify skills 
needs, on site observation and interviews are also performed. Full curriculum 
packages may be requested. 
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3. Employee Orientation 

Presentations were made by the Project Director at all industry sites to 
inform eii5>loyees of the educational program being offered and to motivate them 
to participate. Release time was provided by each company to allow employees 
to attend the orientation sessions • In addition to explaining the procedures 
for participation, the orientation was also directed toward encouraging the 
enployee to take advantage of this opportmity to inprove personally, to enhance 
job success and security and to prepare for future workplace needs. At the 
conclusion of the orientation program, employees are asked to complete a form 
indicating their interest in pursuing the program. Those who indicate interest 
are then advised of the next step in the enrollment process - the individual 
skill level assessment. 
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4 • Assessment 

Although some industry partners chose to require testing plant-wide, in roost 
cases only those individuals expressing interest in program participation were 
tested. The TABE (Test of Adult Basic Skills) locator, a 37-minute 
vocabulary/Daath skill assessment was used. The purpose of the assessment was to 
aid project staff in placing the employee in the educational track roost 
appropriate to his needs. The assessment was professionally administered by 
project counselors and results were individually presented to each en5)loyee. 
With the presentation of scores, the project counselor aided the employee in 
determining his educational program choice and in establishing goals for 
participation. In most cases, this process served as the s' dent enrollment 
procedure, however, a decision to enroll could be made at a later date. 
A copy of the Assessment Result/Enrollment Form is included here. 
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DA.TE: 



T.O.P. ADMISSIONS FORM 



NAME 



ADDRESS 



ss# 



PHO^E: 



BEST TIME TO REACH: 



AGE 



RACE 



SEX 



YRS W/CO. 



SHIFT 



TABE RESULTS! READING 
MATH 



PREV. ENROLLED IN BASIC SKILLS?^ 

HIGHEST GRADE COMPLETED 



PROGRAM OPTIONS SELECTED; 

LITERACY TUTORING 

TRANSPORTATION 



GED 



BASIC SKILLS CLASSES 



CHILD CARE ASSISTANCE 



WHAT DO YOU HOPE TO GET OUT OF THE T.O.P. PROGRAM? 

Indicate eitployee's personally stated goals ^ but also complete checklist. 



Inprove Reading/Language Skills_ 



Improve Math Skill s_ 



Job DeveloEMnent_ 



( Productivity/Advancement) 



Personal DeveloExnent_ 
(Self Esteem) 



COUNSELOR'S COMMENTS 



D:B34 
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5* Training 

Each employee had the option of choosing one of three educational tracks 
offered: 

A* Literacy Tutoring ; Those with no or extremely low reading skills were 
offered a program option of one-on-one tutoring provided through the Greenville 
Literacy Association. In this program, the Laubach method of teaching reading 
to adults is employed. Ihe tutors, who volunteer their time, are matched by the 
Literacy Association to a project participant. Instruction times and locations 
are selected jointly by the student and tutor. 

B. General Educational Developnent Training: Individuals who have not 
conpleted high school could choose to prepare themselves in those skill areas 
needed for GED testing. Classes were established on site to provide this 
coursework, developed by and presented through the State Departinent of 
Education. 

^* ^^sic Literacy Skills Training ; Those enployees not placed in the 
above categories could select basic skill upgrading. This program utlized the 
job specific curriculum developed from the needs analysis. 

For all students the T.O.P. was a voluntary program. Classes (except those 
involving individual tutors) were held at plant sites on off-shift time. 
Tutoring was, as indicated, self -scheduled. 
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6, Support Services 

A. Counseling : The role of the coianselor iDegan with the assessment 
process. As assessment results were provided and enrollments accepted, a 
counselor was assigned to each student. That counselor would assist in 
establishing individual goals for the student at program outset. Interim visits 
to the classroom ensured continual observation of the student and encouragement 
in successful participation. Where absenteeism or other problems arose, plant 
or phone contacts were made and appropriate assistance provided. A copy of the 
counseling record form included here demonstrates the nature of this activity. 

B. Childcare : Assistance with the payment of childcare expenses incurred 
during class time was made available to all program participants. By indicating 
their need for this service and providing their counselor with the childcare 
provision information needed, payment was made to the child care provider on a 
monthly basis. 

C. Transportation s For those students needing transportation assistance in 
order to attend classes, bus passes were provided on a monthly basis. Access to 
this support mechanism was, again, through the program counselor assigned to 
each student. 

The provision of the support services listed above was to ensure that 
traditional barriers to participation would not eliminate those interested in 
pursuing this educational opportunity. 
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T.O-P- 

(TRAINING OPPORTUNITIES PARTNERSHIP) 
Student Counseling torm 



Name 



Program;_ 



Compan y 

Date Entered Program: 



Date of counseling 
Date Initials 



Coinments (be sure to include progress made on checklist 
on back of sheet) 



ERIC 
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?C!?o^ng1reS: ' ''"'^ °' satisfaction or inproven«nt in any of the 



D^TE: 



Reading skills 
Math skills 
Coimiunication skills 
Self-esteem 
Job attendance 
Job productivity 
Personal goals 
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7. Training Completion ; Progress in an adult education curriculum comes 
extremely slowly, yet individual achievement must be recognized and applauded. 
Obviously, those securing a GED received a tangible recognition of their 
coursework coiqpletion. Those in basic skills upgrading or literacy tutoring had 
less documentation of their level of success in the program. For this reason, 
each con^jany established a level of participation that would determine "course 
conqpletion/' That level might be a certain number of classroom hours attended 
pr a more traditional assessment of skill gains or curriculum coitpletion. Based 
on this criteria, certificates of recognition were awarded at each training 
site. A copy of the certificate is included here. 
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For additional information, please contact: 



Joan Mason, Project Director 
Training Opportmities Partnership 
Greenville Technical College 
P. 0. Box 5616 

Greenville, South Carolina 29606 
803-250-8220 
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